Reference Document

Context Engineering

A citable reference on context engineering: the systematic discipline of token budgeting, memory tiering, attention degradation mitigation, prompt caching, and structured payload assembly for LLMs.

Companion Formats & Essays

🎨 ELI5: What Is Context Engineering? — Visual picture-book explainer.
🌳 Context Engineering Knowledge Tree — Prerequisite curriculum map and active frontier.
📖 Essay: The Agent Didn't Get Dumber, the Chat Got Contaminated — Context window drift and contamination.
📖 Essay: Cheap Code, Expensive Context — Economics of token budgeting.

Jump to Section

Definition & Pipeline Thesis · Attention Degradation · Structured Prompt Framework · The Three-Tier Memory Hierarchy · Prompt Caching Mechanics · Compression & Compaction · Context Drift & Contamination · Sources

1. Definition: A Data-Pipeline Discipline

Context Engineering is the systematic design, curation, optimization, and real-time assembly of the token sequence passed to a Large Language Model on each execution turn.

Unlike prompt engineering (which focuses on phrasing, tone, and subjective instructions), context engineering treats context as a strictly constrained, high-latency, expensive computing cache. It is governed as a deterministic ETL (Extract, Transform, Load) data pipeline:

Extract: Dynamically retrieve relevant user history, active files, and semantic RAG vectors.
Transform: Deduplicate passages, prune irrelevant AST branches, compress transcripts, and enforce schema formats.
Load: Pack the token payload within strict budget bounds, ordered to maximize model attention and prompt cache hits.

2. Attention Degradation & "Lost in the Middle"

While modern models support context windows from 128k to 2M tokens, effective recall is non-uniform across the context window.

Empirical evaluations (Liu et al., 2023) show a distinct U-shaped retrieval accuracy curve:

Context Position	Relative Attention Weight	Recommended Payload Content
Primacy (First 15%)	High (>95% recall)	Core system persona, security invariants, behavioral constraints, API schemas, and tool definitions.
Middle (15% – 85%)	Degraded (<60% recall in dense context)	Supporting reference documents, raw RAG search results, background context. (Must be pre-filtered and relevance-ranked).
Recency (Final 15%)	Highest (>98% recall)	Immediate turn task, user input, latest tool execution output, and explicit output formatting rules.

3. The Four-Part Structured Prompt Framework

Production system prompts avoid freeform prose. They partition instructions into four deterministic sections:

Component	Header	Function
1. Role & Identity	`<identity>` / `# Role`	Establishes expertise domain, operational persona, register (e.g. ASD-STE100 technical brevity), and decision authority.
2. Task & Context	`<context>` / `# Task`	Defines the exact objective, relevant environmental state, file paths, and input data.
3. Constraints & Gates	`<rules>` / `# Constraints`	Negative constraints ("Never overwrite originals without confirmation", "Never execute destructive shell commands"), safety thresholds, and reality filters.
4. Output Schema	`<output_format>` / `# Format`	Explicit JSON Schema, XML tag envelope, or tabular structure required for downstream parsing.

4. The Three-Tier Memory Hierarchy

A massive context window cannot replace a persistent, structured memory hierarchy (Mitra, 2026):

Tier	Storage Medium	Lifecycle & Scope	Retrieval Mechanism
Working Memory (Short-Term)	In-memory conversation buffer	Active session turns (sliding window of latest 5–15 turns).	Direct injection into payload.
Episodic Memory (Medium-Term)	Git commit logs, session transcripts, daily journals	Day-to-day session history and task logs.	Time-based search and progressive summarization sweeps.
Semantic Memory (Long-Term)	Curated atomic notes, vector database, Knowledge Trees	Durable domain rules, user preferences, codebase architecture maps.	Vector similarity, keyword grep, and knowledge-graph traversal.

5. Prompt Caching Architecture

Modern frontier inference providers (Anthropic, OpenAI, DeepSeek) support KV (Key-Value) Prompt Caching. When sequential requests share an identical prefix of tokens, the provider reuses the computed attention states from memory rather than re-computing them.

Cost & Latency Gains: Cached tokens are billed at a 75%–90% discount and eliminate up to 80% of Time-To-First-Token (TTFT) latency.
Prefix-Stability Rule: Static tokens (system prompts, tool definitions, reference schemas) must be placed at the absolute start of the context payload. Dynamic tokens (timestamps, conversation turns) must always be appended at the end. Placing a dynamic timestamp at line 1 invalidates the entire downstream cache.

6. Compression, Pruning & Compaction

AST-Guided Pruning: Extracting only relevant class headers, function signatures, and targeted line ranges rather than dumping 5,000-line source files into context.
Progressive Summarization: Distilling raw multi-thousand-word documents into 3-layer atomic notes (Layer 1: raw quotes, Layer 2: bold anchors, Layer 3: executive summary) before context assembly.
Semantic Deduplication: Pruning redundant search results using Maximal Marginal Relevance (MMR) to prevent crowding the context window with repetitive vectors.

7. Context Drift & Chat Contamination

Chat Contamination:

In long multi-turn sessions, intermediate errors, exploratory false starts, and apologetic conversational filler accumulate in the turn buffer. The LLM pays attention to its own past degraded outputs, causing hallucination cascading. Solution: periodic compaction, scratchpad wiping, and fresh-session re-anchoring.

Instruction Drift:

As the context length grows past 50,000+ tokens, the model's adherence to initial system constraints weakens due to distance decay. Mitigated by re-anchoring critical constraints in the recency position immediately before the active user prompt.