Skip to content

Pattern

How human approval works in AI agent systems

Human approval gates sort every action an AI agent can take into three levels: draft only, needs a person to approve, or allowed to run automatically. The agent prepares, a person signs off anything that matters, and the system refuses any action that does not match what was approved.

StudiedLast reviewed 4 min read

Explained from public sources. Mai has not built this yet.

On this page

In plain English

The safest useful AI is one that does the legwork and leaves the decisions to you. Approval gates make that a rule rather than a hope. Every action an agent can take is put in one of three boxes: things it may only draft, things that wait for a person to say yes, and small, safe things it may do on its own. When an action needs approval, the agent stops, shows exactly what it wants to do, and carries on only once someone has decided.

How it works

Three levels, set per action. The level belongs to the action and its context, not to the agent:

  • Draft. The agent prepares, a person does. It writes the reply; you send it.
  • Approve. The agent will do it, but only after a named person approves this specific action. Sending to a client, booking a contractor, updating a tenancy record.
  • Automatic. Low-risk, reversible, internal actions run straight away and are logged: filing an email, adding a calendar hold, creating a task.

The same tool can sit at different levels depending on its arguments. An internal email to a colleague might be automatic; the same tool addressed outside the company needs approval. A payment under a set amount to a known supplier might need one approver; anything above it might need two.

Pause, show, resume. When a gated action comes up, the system saves the run’s state and the exact proposed action, sends it to the approver in a channel they already use, and stops. Frameworks support this directly. LangGraph, for example, has an interrupt that suspends the run, saves its state with a checkpointer and resumes when the application passes back a decision, and the decision can approve, edit or cancel a tool call before it runs. Anthropic’s guide to building agents likewise describes agents pausing at checkpoints for human judgement.

Bind the approval to the action. A plain “yes” is not enough. The approval should be tied to precisely what was shown: this recipient, this text, this amount. A common way is to sign a token over the action’s details. The tool layer checks the token before executing and refuses if anything has changed since approval, if the token has already been used, or if it has expired.

Agent proposes. Then Policy check. Then You approve. Then Token verified, Action runs. Then Audit log.

The agent prepares, a person decides, and only an action matching the approval runs

Why it is not optional. OWASP lists excessive autonomy, letting high-impact actions run without independent checks, as one of three root causes of agents causing harm, and recommends human approval for high-impact actions alongside narrow permissions and having downstream systems authorise every request themselves. The MCP specification similarly recommends that a person can always deny a tool call and that clients ask for confirmation on sensitive operations.

Where it goes wrong

  • Rubber-stamping. If everything needs approval, people approve without reading. Gates should be rare enough to be taken seriously, and the approval screen should show the decision in one glance: what, to whom, why, and what the agent based it on.
  • Approving one thing, running another. If the agent can change the draft after approval, the gate is theatre. Binding the approval to the exact action closes this.
  • Gates enforced only in the prompt. “Always ask before sending” in an instruction can be argued away or injected around. The check has to live in code, in the tool layer.
  • Stuck work. Approvals that sit unanswered stall the task. Requests need an owner, a reminder and an expiry, plus a fallback approver for holidays.
  • No record of the decision. Who approved what, when and on what evidence must be logged, or you cannot answer for the action later (see traces and audit logs).
  • Never promoting anything. If an action is approved unchanged every time for months, keeping it gated wastes attention. Moving it to automatic should be a deliberate, logged decision.

How Mai AI uses it

Approval gates are the rule I start every design from. In the AI chief of staff, every action an agent takes is checked against what it is allowed to do, anything that matters needs a signed approval token, and the whole trail is written to an audit log. The agents prepare, draft and chase; the decisions wait for a person.

In a Mai AI system, I agree the action list with the client at the start: every write tool, its default level, who approves it and any thresholds. The policy lives in the tool layer, not in prompts, so no agent can skip it, however it is instructed. Approvals reach people where they already work, and each one shows the evidence behind the request.

Levels can change as trust builds, but only by a decision someone makes and the log records. If you want to see where the gates would sit in your own processes, that is a core part of designing your custom AI operations system.

Questions and answers

What is human in the loop for AI agents?

It means a person reviews and approves certain actions before an agent carries them out, such as sending an email or changing a record. The agent pauses, shows what it intends to do, and continues only after a decision.

Which AI agent actions should need human approval?

Anything that is hard to undo, leaves the business, spends money or changes important records: sending external messages, payments, deletions and contract changes. Reading and drafting can usually run without approval.

How does an AI agent wait for approval?

The system saves the agent state and the exact proposed action, notifies the approver, and stops. When the person decides, the run resumes from the saved state and either carries out the approved action or cancels it.

Do approval gates slow AI agents down?

A little, by design, but only where it matters. Because the agent has already done the gathering and drafting, approving is usually a quick check, and low-risk actions can be moved to automatic once they have a good track record.

Sources

  1. LLM06:2025 Excessive Agency (OWASP GenAI Security Project)genai.owasp.org
  2. Tools (Model Context Protocol specification)modelcontextprotocol.io
  3. Interrupts (LangGraph documentation)docs.langchain.com
  4. Building effective agents (Anthropic engineering)anthropic.com