Systems Design · AI Safety
Plan-Approve-Execute
Think first, ask for permission, then do the work.
1. The runaway helper
If you give an AI power to press big red buttons immediately, small mistakes turn into disasters.
Why does this happen?
AI models solve problems step by step. If an agent has write access while reasoning, a hallucinated assumption will trigger real destructive commands before you can stop it.
2. The three-step safety rule
First write the plan on paper, ask for a signature, and then run only what was approved.
What tools can it use in Step 1?
During planning, the agent only has read-only tools like viewing files, searching documentation, and checking system status. It cannot modify anything.
3. Clear tool boundaries
Reading information is always allowed; touching live systems requires explicit human permission.
What if an agent tries to bypass the gate?
The control plane blocks unauthorized tool calls immediately. The agent cannot run commands without an active approval ticket.
Longer version: Reference Entry on Plan-Approve-Execute
Sources: Mitra, System Design for the LLM Era (2026); OWASP LLM08 Excessive Agency.