3  The formal definition

Author

Will Landau

Building on the intuition from the previous chapter, this chapter formalizes the concept of trusted mini-agents. It introduces a formal definition, interprets it in terms of the fixes we made to the weather agent, grounds it in simple mathematical concepts, and describes the real-world workflow of a trusted mini-agent.

3.1 Defining trusted mini-agents

NoteDefinition: Trusted Mini-Agent

A trusted mini-agent is a least-privilege AI agent that engineers out AI errors with the following three rules:

  1. Trusted tools directly produce all results. No result comes from the AI model. And for each type of result, there exists a tool to produce it.
  2. Each result only comes from one trusted tool. No alternative tool can bypass the designated tool.
  3. Inputs to trusted tools have trusted human oversight. The system presents AI-generated tool input in a form that humans can verify quickly and accurately, with minimal time and minimal fatigue.

The three rules map to the three fixes we made to the weather agent.

Rule Fix
1. Trusted tools directly produce all results. Bypass the AI model to report the weather.
2. Each result only comes from one trusted tool. Never bypass the get_weather() tool.
3. Inputs to trusted tools have trusted human oversight. Show the location in an interactive map.

In terms of tools and results, a trusted mini-agent is a bijective function. Rule 1 implies surjectivity: every result can be produced by at least one tool. Rule 2 implies injectivity: each result must originate from at most one designated tool. Here is a mental model of bijectivity for an enhanced weather tool that also reports emergency alerts:

\[ \Large f: X \mapsto Y \]

%%{init: {'theme':'base', 'themeVariables': {'lineColor':'#1F3A63','edgeLabelBackground':'#ffffff','fontSize':'18px'}}}%%
flowchart LR
    subgraph X["X: tools"]
        direction TB
        gw["get_weather()"]
        gn["get_emergency_alerts()"]
    end
    subgraph Y["Y: types of results"]
        wd["Weather data"]
        nd["Emergency alert data"]
    end
    gw --> wd
    gn --> nd

    style gw fill:#1F3A63,color:#ffffff,stroke:#1F3A63
    style gn fill:#1F3A63,color:#ffffff,stroke:#1F3A63
    style wd fill:#1F3A63,color:#ffffff,stroke:#1F3A63
    style nd fill:#1F3A63,color:#ffffff,stroke:#1F3A63
    style X fill:#DCEAF7,stroke:#9CA3AF,color:#1F3A63
    style Y fill:#DCEAF7,stroke:#9CA3AF,color:#1F3A63

Figure 3.1: Diagram illustrating a bijective function \(f\) from \(X\) to \(Y\). The set \(X\) (tools) contains get_weather() and get_emergency_alerts(); the set \(Y\) (types of results) contains Weather data and Emergency alert data. Each tool maps to exactly one type of result, and each type of result is produced by exactly one tool, illustrating Rules 1 and 2 of a trusted mini-agent.

Bijectivity is key: it ensures that final results are trusted as long as the tools and their inputs are trusted. Tools are easy to test with expectation-based unit tests. Inputs to those tools are harder to check because they depend on user oversight, but verifying inputs is usually much easier than verifying outputs.

3.2 Impact on users

Instead of:

Are these results correct?

users ask:

Is the agent solving the right problem?

which is much easier to verify.

3.3 The trusted mini-agent workflow

The trusted mini-agent workflow combines the power of modern agents with the safeguards of traditional software engineering:

%%{init: {'theme':'base', 'themeVariables': {'lineColor':'#4B5563','edgeLabelBackground':'#ffffff'}}}%%
flowchart LR
    subgraph AgentLoop["Agent Loop"]
        direction TB
        Model["Model"]
        UT["Untrusted<br>Tools"]
        Model -->|"Autonomous<br>reasoning"| UT
        UT --> Model
    end

    HR["Human<br>Review"]

    subgraph Trust["Trust"]
        direction TB
        TT["Trusted<br>Tools"]
        TR["Trusted<br>Results"]
        TT -->|"Nontrivial<br>Computation"| TR
    end

    AgentLoop -->|"Proposed Inputs"| HR
    HR -->|"Rejected Proposals"| AgentLoop
    HR -->|"Reviewed Inputs"| TT

    style Model fill:#1F3A63,color:#ffffff,stroke:#1F3A63
    style UT fill:#6B2020,color:#ffffff,stroke:#6B2020
    style HR fill:#166534,color:#ffffff,stroke:#166534
    style TT fill:#166534,color:#ffffff,stroke:#166534
    style TR fill:#166534,color:#ffffff,stroke:#166534
    style AgentLoop fill:#FDE8E8,stroke:#9CA3AF
    style Trust fill:#F3F4F6,stroke:#9CA3AF

Figure 3.2: Flow diagram of a trusted mini-agent. On the left, an “Agent Loop” box contains the AI model and Untrusted Tools connected by bidirectional arrows labeled “Autonomous reasoning.” The agent loop sends “Proposed Inputs” to a “Human Review” checkpoint, which can return “Rejected Proposals” back to the loop. After review, “Reviewed Inputs” flow into “Trusted Tools” inside the “Trust” boundary, which perform “Nontrivial Computation” and produce “Trusted Results” below.

After an initial prompt, the work begins with the agent loop. Here, the AI model ponders and deliberates uninterrupted and unsupervised, potentially with the aid of untrusted tools (e.g. web search, back-of-the-envelope calculations, etc.) which are structurally incapable of producing final results.

Ultimately, the AI model proposes inputs to one or more trusted tools that perform critical computations such as modeling and simulation. Those inputs reflect how the AI model frames up the problem and sets up an advanced computation. Since they are about overall framing and human intent, they are easy to check for AI errors, especially if the system for oversight is carefully structured, with realistic expectations about human understanding, attention, and fatigue.

After a trusted system of review, the AI-generated inputs move to a set of designated trusted tools that cannot be bypassed. Those trusted tools perform important computations such as statistical analysis, modeling, and simulation. Results from such computations are usually too complex for humans to check, but they are guaranteed to come from trusted tools on verified inputs. Conditional on good inputs, AI-generated errors in the results are structurally impossible.

3.4 When trusted mini-agents are useful

Compared with the sweeping power of coding agents and multi-agent RAG pipelines, trusted mini-agents may seem anticlimactic or underwhelming. However, they excel in precision scenarios where the task is narrowly defined and human oversight is feasible. The hardest part is framing the problem. The goal is to insert the AI model where:

  1. The AI model proposes solutions better than a human.
  2. A human easily checks those solutions.

No scenario is too small for a trusted mini-agent, and specific, targeted problems offer some of the most powerful opportunities. For example, clinical trial simulation offers powerful opportunities, as discussed later in Chapter 6.