← Knowledge Trees · Nestor G Pestelos Jr

LLM System Design Knowledge Tree

Prerequisite curriculum: token mechanics, context pipelines, RAG graphs, gateways, resilience, and evaluation.

Target Capability

Architect production-grade LLM applications: select models along the 4-tier capability spectrum, construct deterministic context pipelines, integrate hybrid vector/graph RAG retrieval, deploy resilient reverse-proxy gateways with tiered fallbacks, and validate systems with automated LLM-as-a-judge evals.

Filter view:

Phase 1 — Fundamental Units & Model Selection 3 / 3 Mastered

Mastered §1.1

Tokens & Dense Vector Embeddings

Test: Explain subword tokenization; demonstrate why embeddings cluster by conceptual meaning rather than string equality using cosine distance.

Prerequisites: None (Domain Entry)
Inspect Evidence & Links
Evidence: Authored atomic note "Embeddings Cluster by Meaning Not Exact Match" (2026-08-23).
Mastered §1.2

The Four-Tier Model Selection Spectrum

Test: Route four distinct production workloads (planning, coding, triage, local privacy) to Frontier, Workhorse, Flash, and SLM tiers with cost-latency justification.

Prerequisites: Tokens & Dense Vector Embeddings
Inspect Evidence & Links
Evidence: Authored atomic note "The Four-Dimension LLM Selection Trade-off" (2026-08-23).
Mastered §1.3

Multimodal Processing Tiers

Test: Classify text, vision, audio, and structured data pipelines into token cost and latency implications.

Prerequisites: The Four-Tier Model Selection Spectrum
Inspect Evidence
Evidence: Authored atomic note "Five AI Input Modalities Beyond Text" (2026-08-23).

Phase 2 — Context Engineering & Prompts 3 / 3 Mastered

Mastered §1.4

Four-Part Structured Prompt Framework

Test: Construct a deterministic system prompt partitioned into Identity/Role, Task/Context, Safety/Constraints, and Output Schema.

Prerequisites: The Four-Tier Model Selection Spectrum
Inspect Evidence
Evidence: Authored atomic note "The Four-Part Structured Prompt Framework" (2026-08-23).
Mastered §1.5

Context Overflow & Memory Hierarchy

Test: Design a memory hierarchy (Working Turn Buffer → Episodic Logs → Semantic Notes) that prevents "Lost in the Middle" degradation.

Prerequisites: Four-Part Structured Prompt Framework
Inspect Evidence
Evidence: Authored atomic note "Context Overflow Needs a Memory Strategy Not Just a Bigger Window" (2026-08-23).
Mastered §1.6

Context Engineering as a Data Pipeline

Test: Implement prompt compression, token-budget enforcement, and dynamic data packing before model invocation.

Prerequisites: Context Overflow & Memory Hierarchy
Inspect Evidence
Evidence: Authored atomic note "Context Engineering Is a Data-Pipeline Discipline Not a Prompting Skill" (2026-08-23).

Phase 3 — Code Understanding & Hybrid RAG 3 Mastered, 1 Frontier

Mastered §1.7

AST Chunk Boundaries

Test: Contrast fixed-character chunking with Abstract Syntax Tree (AST) class/function chunk boundaries for codebases.

Prerequisites: Tokens & Dense Vector Embeddings
Inspect Evidence & Links
Evidence: Authored atomic note "ASTs Define Codebase Chunk Boundaries at the Class or Function Level" (2026-08-23).
Mastered §1.8

Vector Retrieval vs. Redundant Similarity

Test: Explain why naive top-K cosine similarity returns redundant chunks, and implement Maximal Marginal Relevance (MMR).

Prerequisites: AST Chunk Boundaries
Inspect Evidence
Evidence: Authored atomic note "Naive RAG Buries Answers Under Redundant Similarity" (2026-08-23).
Mastered §1.9

GraphRAG & Relational Queries

Test: Combine knowledge-graph entity traversal with vector similarity to resolve multi-hop dependency questions.

Prerequisites: Vector Retrieval vs. Redundant Similarity
Inspect Evidence & Links
Evidence: Authored atomic note "GraphRAG Combines Knowledge-Graph Traversal With Vector Retrieval" (2026-08-23).
Frontier §2.1

RAG Ingestion Pipelines

Test: Design an end-to-end document parsing, deduplicating, embedding, and tenant-isolated ingestion pipeline.

Prerequisites: GraphRAG & Relational Queries

Phase 4 — Production Architectural Patterns Locked

Locked §2.2

The LLM Gateway Pattern

Test: Build a reverse-proxy gateway abstracting vendor SDKs, normalizing schemas, and injecting tracing headers.

Prerequisites: Context Engineering as a Data Pipeline
Locked §2.3

Circuit Breakers & Tiered Fallbacks

Test: Wire automated failover from primary to secondary LLM endpoints when error rates exceed threshold.

Prerequisites: The LLM Gateway Pattern
Locked §2.4

Three-Level LLM Caching

Test: Implement exact-match hash cache and semantic embedding cache with collision prevention.

Prerequisites: The LLM Gateway Pattern

Phase 5 — Evaluation, Security & Safety Locked

Locked §2.5

LLM-as-a-Judge Golden Set Benchmarking

Test: Construct a golden dataset and automated evaluation harness scored by a rubric-constrained judge model.

Prerequisites: The LLM Gateway Pattern
Locked §2.6

Firewall LLM & Plan-Approve-Execute Gating

Test: Implement pre-call intent filtering and separate execution policy from generation logic.

Prerequisites: LLM-as-a-Judge Golden Set Benchmarking