3 Defining trusted mini-agents
In the previous chapter we improved the weather agent by restricting its capabilities. We applied the principle of least privilege to the model, restricting its influence to hardened components we can either test in advance or verify at runtime. The trusted mini-agent design pattern generalizes this idea to all agents.
3.1 Definition
A trusted mini-agent is an AI agent that follows these three rules:
- Every output of the AI model is untrusted. Model output is always suspicious, even if it appears to simply echo the output of a tool.
- Every result is the direct output of a tool trusted to produce it. A result is an output the agent exists to produce, such as a calculation, a dataset, or a diagram. Conversation along the way is not a result. Given correct inputs, a trusted tool correctly produces the results it is trusted for: those results are correct and need no further verification.
- Every input to a trusted tool has trusted human oversight. The system presents the human user with the inputs to trusted tools. The human approves or rejects those inputs, and no result is trusted until the inputs that produced it are approved. The oversight process itself is trusted when the system presents inputs in a form that a human can judge accurately, quickly, and without fatigue. Inputs to untrusted tools need no such review.
3.2 Example
The three rules map to the three fixes we made to the weather agent.
| Rule | Fix |
|---|---|
| 1. Every output of the AI model is untrusted. | Bypass the AI model to report the weather. |
| 2. Every result is the direct output of a tool trusted to produce it. | Never bypass the get_weather() tool. |
| 3. Every input to a trusted tool has trusted human oversight. | Show the location in an interactive map. |
3.3 Untrusted tools
An agent may support untrusted tools, as long as those tools do not produce key results. Tools such as web search or the ClinicalTrials.gov API can inform the AI model’s reasoning without being trusted for any particular result. Their output feeds back into the agent loop, where by rule 1 the model’s reasoning is already suspicious. Because these tools do not produce results, rule 3 does not require oversight of their inputs. Human oversight begins downstream, when the AI model proposes inputs to a trusted tool.
3.4 Oversight
The trusted human oversight in rule 3 can take many different forms.
3.4.1 Gating
Some agents require the human to approve inputs before a tool runs, while others run the tool eagerly and prompt the human to verify post hoc. The weather agent, simulation agent, commons, clinical-trials-agent, and retroglyph all use post hoc oversight. Gated review is more akin to manual mode in Claude Code.
3.4.2 Projections
A trusted mini-agent seldom parrots verbatim tool inputs for the user to check. Instead, it offers a friendly readable projection of those inputs that a human can verify at a glance. The weather agent, rather than listing latitude and longitude, drops a pin on a map. retroglyph, rather than regurgitating the raw color codes that feed its tools, overlays a reconstructed image on the target Kaplan-Meier image so discrepancies stand out. Although that reconstructed image is derived from tool outputs (i.e. the reconstructed individual patient data), it is still a projection of the inputs: comparing it against ground truth (the target image) verifies that the inputs were correct.