Agent Harness

Reference entry · last updated 20260915

Previous version, captured 20260915

An agent harness is the software and runtime environment surrounding a foundation model that manages its execution. It supplies tools, context, state, feedback, and permission controls.[1]

First principles: a computer architecture analogy

Beren Millidge compares scaffolded language models to a computer. The mapping is an analogy: language-model outputs lack the reliability of CPU instructions.[3]

LLM ≈ CPU: Transforms input tokens into output tokens.
Context window ≈ RAM: Holds the prompt and information available for a model call.
External memory ≈ Disk storage: Stores information beyond the current context.
Tools ≈ Device drivers: Connect the model to external systems.
Scaffolding protocols ≈ Programs: Chain model calls into a larger task.[3]

Harness changes and measured performance

In a 20260217 report, LangChain's Vivek Trivedy described improving deepagents-cli from 52.8% to 66.5% on Terminal Bench 2.0 while keeping GPT-5.2-Codex fixed. The team changed the harness, including prompts, tools, and middleware, and adjusted reasoning budgets.[4]

This result shows an improvement for that agent and benchmark. It does not establish that harness changes generally outperform model upgrades, or isolate verification as the cause of the gain.

Twelve architectural components

These twelve items group harness responsibilities from the cited designs for illustration. A system may combine them or omit features it does not need.

Orchestration loop: Calls the model, executes tools, and returns their results for the next step.[1]
Tool interface: Defines available actions and their inputs. Anthropic recommends clear tools with little overlap.[2]
Memory layer: Stores notes outside the context window and retrieves them when needed.[2]
Context management: Summarizes older conversation and clears redundant tool results. Compaction can lose details that later matter.[2]
Prompt assembly: Selects system instructions, tool descriptions, external data, and message history for each call.[2]
Input/output handling: Passes requests to the model and routes tool calls to execution infrastructure.[1]
State persistence: Uses progress files and Git history to help later sessions resume work.[5]
Error handling: Provides recovery paths after failures, such as restoring a working Git state.[5]
Guardrails: Uses hooks and sandbox restrictions to enforce action boundaries.[1]
Verification loops: Runs tests and returns failures to the agent. In Anthropic's web-app experiments, browser testing exposed defects that code inspection missed.[5]
Subagent management: Delegates focused tasks to separate contexts and collects their results.[2]
Execution flow: Controls when work continues or stops, including checks before completion.[4]

Skills and runtime responsibilities

Agent Skills package task instructions in SKILL.md files and can include scripts and reference material. Compatible agents load them when relevant.[6]

Runtime code enforces permissions, invokes tools, and runs checks at defined lifecycle points. A written instruction to avoid an action is not an enforcement boundary.[1]

Skills make procedures easier to reuse, but format compatibility does not ensure identical behavior. Millidge notes that model-specific prompts and failsafes limit portability; LangChain likewise recommends tuning harnesses for the model and task.[3, 4] Portability therefore needs testing in the destination environment.

References

↑ Osmani, Addy. “Agent Harness Engineering.” 20260419.
↑ Anthropic Applied AI team. “Effective context engineering for AI agents.” 20250929.
↑ Millidge, Beren. “Scaffolded LLMs as natural language computers.” 20230411.
↑ Trivedy, Vivek. “Improving Deep Agents with harness engineering.” LangChain, 20260217.
↑ Young, Justin. “Effective harnesses for long-running agents.” Anthropic, 20251126.
↑ Agent Skills. “Agent Skills Overview.” Accessed 20260915.