← Reference · Nestor G Pestelos Jr

Reference Document

Agents

A citable ground-truth reference on Autonomous Agents: MDP formulation, perception-action-reflection loops, tool calling protocols, memory tiers, and multi-agent orchestration.

1. Formal Definition & Mathematical Model

An AI Agent is an autonomous computational system that perceives its environment through multimodal sensors/context inputs, maintains internal state and goals, formulates sequential reasoning plans, and executes discrete actions via external tools to affect environmental state across multi-turn trajectories.

Formally, an agent operates within a Partially Observable Markov Decision Process (POMDP) defined by the tuple:

• $\mathcal{S}$: Environmental state space (e.g., codebase filesystem, database records, API server state).
• $\mathcal{A}$: Discrete action space parameterized by structured tool calls: $a_t = (\text{tool\_name}, \text{arguments})$.
• $\mathcal{O}$: Observation space (e.g., shell stdout/stderr, API responses, compiler outputs).
• $\mathcal{T}(s_{t+1} \mid s_t, a_t)$: State transition probability density function governed by the external system.
• $\mathcal{Z}(o_t \mid s_t, a_{t-1})$: Observation emission function mapping system state to tokenized observations.
• $\mathcal{R}(s_t, a_t)$: Reward function (or binary evaluation success criteria $\mathcal{M}(s_{\text{final}}) \in \{0, 1\}$).
• $\gamma \in [0, 1]$: Temporal discount factor for multi-turn planning horizon.

The agent parameterizes an autoregressive policy $\pi_\theta(a_t \mid h_t)$ conditioned on the cumulative trajectory history $h_t = (o_0, a_0, o_1, a_1, \dots, o_t)$, optimizing expected cumulative utility:

2. The Four Core Agent Subsystems

3. Execution Loops (ReAct, Reflexion, Plan-and-Solve)

4. Tool Calling Protocols & Sandboxing

Modern agents interact with host operating systems via standardized tool calling protocols (e.g., Anthropic Tool Use API, OpenAI Function Calling, MCP — Model Context Protocol).

// Formal Tool Schema Specification (JSON Schema)
{
  "name": "replace_file_content",
  "description": "Edits an existing file via exact block match replacement.",
  "parameters": {
    "type": "object",
    "properties": {
      "target_file": { "type": "string", "description": "Absolute path" },
      "target_content": { "type": "string", "description": "Exact text to replace" },
      "replacement_content": { "type": "string", "description": "New content" }
    },
    "required": ["target_file", "target_content", "replacement_content"]
  }
}

Sandboxing & Execution Safety:

• Process Isolation: Running arbitrary shell and interpreter execution inside ephemeral containers (Docker/gVisor), Linux namespaces, or isolated WASM runtimes.
• Path Traversal & Permission Gates: Strict allowlisting of workspace directories and explicit human approval gates for destructive commands (e.g., rm -rf, DROP TABLE, cloud infrastructure provisioning).
• Idempotent Operations: Designing tools such that repeated invocation does not cause corrupting side effects (e.g., git patch verification, atomic writes).

5. Multi-Agent Topologies & Orchestration

Complex workflows exceed the context window and cognitive capacity of a single monolithic agent. Multi-agent systems distribute work across specialized agent nodes.

6. Failure Modes & Defensive Guardrails

7. Architecture Comparison Matrix