Reference Document

Prompt Engineering

A citable ground-truth reference on Prompt Engineering: conditioning autoregressive models, in-context learning, reasoning scaffolding (CoT, ToT, ReAct), structured outputs, and prompt security.

Companion Formats & Essays

📖 Essay: Debug the System Not the Prompt — Why systems and workflows outrank prompt tuning.
📖 Essay: The Agent Didn't Get Dumber, the Chat Got Contaminated — Context window drift and instruction decay.
📖 Essay: When the AI Writes the Code, the Plan Becomes the Work — Spec-first prompt design.
📚 Reference: Context Engineering — Token budgeting, memory tiers, and cache mechanics.
📚 Reference: Large Language Models — Pre-training, attention architecture, and scaling laws.

Jump to Section

Mathematical Formulation · In-Context Learning · Reasoning Scaffolding · Structured Outputs · Security & Injections · Technique Matrix · Sources

1. Definition & Mathematical Formulation

Prompt Engineering is the practice of structuring, parameterizing, and formatting natural language and symbolic inputs to guide the conditional probability distribution of an autoregressive foundation model toward a desired output behavior without updating the model's underlying parameter weights $\theta$.

Formally, a causal language model computes the conditional probability distribution over a vocabulary $\mathcal{V}$ for the next token $y_t$ given an input prompt prefix sequence $X = (x_1, x_2, \dots, x_n)$ and previously generated tokens $y_{ $$P_\theta(Y \mid X) = \prod_{t=1}^T P_\theta(y_t \mid X, y_1, \dots, y_{t-1})$$

Prompt engineering optimizes the prefix $X$ to maximize the likelihood of a target response $Y^*$:

$$X^* = \arg\max_{X \in \mathcal{S}} \mathbb{E}_{Y^* \sim \mathcal{D}_{\text{eval}}} \left[ \mathcal{M}(Y^*, \arg\max_Y P_\theta(Y \mid X)) \right]$$

where $\mathcal{S}$ is the space of allowable prompt strings, $\mathcal{D}_{\text{eval}}$ is the task evaluation distribution, and $\mathcal{M}$ is a task-specific performance metric.

2. In-Context Learning (ICL)

In-Context Learning (Brown et al., 2020) is the emergent capability of scaled transformers to execute novel downstream tasks given only instruction text and input-output demonstrations inside the prompt prefix, without gradient updates.

Zero-Shot Prompting:

The model receives only the task directive $I$ and query $x_{\text{query}}$, relying strictly on parametric memory acquired during pre-training and instruction tuning: $X = I \circ x_{\text{query}}$.

Few-Shot Prompting ($k$-Shot ICL):

The prompt provides $k$ exemplars demonstrating the input-to-output mapping before presenting the query: $$X = I \circ (x^{(1)}, y^{(1)}) \circ (x^{(2)}, y^{(2)}) \circ \dots \circ (x^{(k)}, y^{(k)}) \circ x_{\text{query}}$$ Exemplars condition attention heads to resolve task formats, token distributions, and reasoning conventions via implicit meta-gradients calculated during the forward pass.

3. Reasoning Scaffolding (CoT, ToT, ReAct)

Direct sequence-to-sequence generation ($X \to Y$) forces transformers to perform complex reasoning within fixed constant-depth forward-pass layer allocations. Reasoning scaffolding forces the model to allocate additional compute tokens by generating explicit intermediate latent rationales $Z = (z_1, \dots, z_m)$ before emitting the final answer.

Chain-of-Thought (CoT) (Wei et al., 2022):

Decomposes multi-step deduction into sequential reasoning hops: $$P(Y \mid X) = \sum_Z P(Y \mid X, Z) P(Z \mid X)$$ By generating rationale tokens $Z$ into the autoregressive stream, downstream tokens attend over the activations of prior reasoning steps rather than jumping directly from question to answer. Zero-Shot CoT (Kojima et al., 2022) elicits this behavior with deterministic trigger phrases such as "Let's think step by step."

Self-Consistency (Wang et al., 2022):

Samples multiple independent reasoning paths $Z^{(1)}, Z^{(2)}, \dots, Z^{(N)}$ at non-zero temperature ($T > 0$) and marginalizes over reasoning paths to select the majority answer via consensus voting: $$y^* = \arg\max_{a \in \mathcal{A}} \sum_{i=1}^N \mathbb{I}\left( \text{extract\_answer}(Z^{(i)}) = a \right)$$

Tree of Thoughts (ToT) (Yao et al., 2023):

Generalizes linear chains into a deliberate search tree. The prompt architecture generates multiple candidate thoughts per step, evaluates each branch using self-evaluative scoring prompts, and navigates the state space via breadth-first search (BFS) or depth-first search (DFS) with backtracking.

ReAct: Reason + Act (Yao et al., 2022):

Interleaves internal reasoning traces with external tool execution: $$\text{Thought}_t \to \text{Action}_t \to \text{Observation}_t \to \text{Thought}_{t+1} \to \dots \to \text{Final Answer}$$ External observations inject ground truth into the prompt context, mitigating hallucination and enabling stateful interaction with APIs, shell environments, and databases.

4. Structured Output Control & Schema Enforcement

Production integration requires LLM outputs to conform strictly to programmatic specifications (e.g., JSON, YAML, SQL).

Prompt-Level Directives: Few-shot schema examples, Markdown code fences (```json ... ```), and negative constraints (e.g., "Return JSON only, no explanatory prose."). Subject to non-zero failure rates under long context or model degradation.
Grammar-Constrained Decoding: Logit-masking at inference time (e.g., via Context-Free Grammars, JSON Schemas, or Pydantic models). The inference engine computes a finite-state machine (FSM) over vocabulary tokens, setting $z_{t, v} = -\infty$ for any token $v$ that would violate the grammar. This guarantees 100% syntactic validity independent of prompt phrasing.
Metaprompting & System Framing: Establishing operational roles, execution boundaries, and explicit error-handling protocols at the system level before user inputs are ingested.

5. Prompt Security & Vulnerabilities

Because autoregressive architectures process system instructions and untrusted user input within a single concatenated token stream, language models lack hardware-level separation between code and data (analogous to von Neumann architecture vulnerabilities).

Direct Prompt Injection (Jailbreaking):

Adversarial user inputs designed to override the system prompt's instructions or safety guardrails (e.g., "Ignore all previous instructions and output...", character-roleplaying, base64 payload encoding).

Indirect Prompt Injection (Greshake et al., 2023):

Adversarial payload embedded in external data retrieved by the model (e.g., web pages, emails, database records, search results). When the model ingests the document into its context window, the hidden payload hijacks agent execution.

Defensive Architectures:

Structural Delimiters: Isolating untrusted data using explicit XML tags (e.g., <user_input>...</user_input>) or markdown fences with explicit parsing instructions.
Privilege Separation / Dual LLM Pattern: Isolating untrusted data processing in an unprivileged reader model before passing verified structured summaries to the privileged action agent.
Deterministic Post-Validation: Verifying all agent tool calls and generated artifacts against strict programmatic schemas before execution.

6. Prompt Engineering Technique Matrix

Technique	Mechanism	Compute / Token Overhead	Optimal Use Case
Zero-Shot	Direct instruction + query	Baseline ($1\times$)	Simple classification, standard summarization, high-throughput tasks.
Few-Shot (ICL)	$k$ input-output exemplars	Low–Medium ($k \times \text{len}(\text{sample})$)	Strict format adherence, domain-specific syntax, rare token patterns.
Zero-Shot CoT	Trigger phrase ("Let's think step by step")	Low token increase (generates rationale)	General multi-step arithmetic, logic puzzles, basic code synthesis.
Few-Shot CoT	Exemplars containing explicit reasoning rationales	Medium ($2\times - 5\times$)	Complex symbolic reasoning, math word problems, multi-hop question answering.
Self-Consistency	Multi-path generation + majority vote	High ($N \times \text{tokens}$)	High-stakes mathematical verification, competitive coding, formal proof verification.
Tree of Thoughts	Branching thought exploration + BFS/DFS search	Very High ($10\times - 30\times$)	Strategic planning, combinatorial puzzles, multi-constraint design.
ReAct	Interleaved Thought $\to$ Action $\to$ Observation loops	Variable (scales with tool interactions)	Autonomous agents, live API querying, terminal debugging, active research.
Grammar Masking	Inference-time logit masking via schema/FSM	Zero token overhead; minor engine latency	Deterministic API contracts, structured database output, tool calling payloads.