Pattern
How routing and specialist AI agents work
Routing means one agent reads each request and hands it to the specialist best placed to handle it, such as finance, operations or sales. Each specialist has its own instructions, tools and permissions, which keeps answers sharper and access tighter than one agent trying to do everything.
Explained from public sources. Mai has not built this yet.
On this page
In plain English
Think of a good office manager. They do not answer every question themselves; they know who in the building handles invoices, who handles maintenance and who handles new clients, and they pass each request to the right person with a short note. A routed agent system works the same way. One agent, the router, reads what has come in and decides who should deal with it. Specialist agents then do the work in their own area, using only the tools and information that area needs.
How it works
The router. The router’s job is classification, not problem solving. It reads the request (an email, a Teams message, a form) along with a little context, such as who sent it, and picks a destination. It can be a language model, simple rules, or both: rules catch the obvious cases cheaply and the model handles the ambiguous ones. Anthropic’s guide to building agents describes routing as classifying an input and sending it to a specialised follow-up, so that each category can have its own prompt and tools, and notes that the same idea can send easy requests to smaller models and hard ones to more capable models.
The specialists. Each specialist agent has:
- its own instructions, written for its area (how this business handles arrears, say, rather than generic advice);
- its own tools, limited to the systems its area uses;
- its own permissions, so the sales agent cannot read payroll;
- its own evaluation cases, so it can be tested on its own.
The handoff. When the router passes work on, it sends a structured note: what was asked, who asked, why this specialist, and any facts already gathered. In the OpenAI Agents SDK, for example, each possible handoff appears to the model as a tool named after its destination, and the developer can filter how much of the earlier conversation the receiving agent sees. Deciding what to pass is a design choice: the full history is simplest, a filtered summary is cheaper and leaks less.
In: Email, Teams, Forms. Then Router. Then Finance, Operations, Sales, People. Then Own tools and data. Then You approve.
Some requests span areas: “can we afford to take on this contract?” needs finance and operations. The router then either sends it to a coordinating agent that consults both, or asks each specialist for its part and combines the answers. That is where routing meets sub-agents and parallel work.
Routing also helps security. OWASP’s guidance on excessive agency recommends giving each part of a system only the functions and permissions it needs. Splitting work by area is a natural way to do that: the blast radius of any one agent is its own area.
Where it goes wrong
- Misrouting. The router sends a complaint to sales or a supplier invoice to the people team. Every route needs a fallback (“not sure, ask a person”) and a log of routing decisions, so mistakes are visible and can become test cases.
- Too many specialists. Ten agents with overlapping remits confuse the router and multiply maintenance. Start with a few areas that match how the business is actually organised.
- Lost context. If the handoff note is thin, the specialist asks the customer the same questions again. If it is too full, it carries data the specialist should not see.
- Ping-pong. Specialists handing work back and forth without anyone owning it. One agent must own each task until it is done or escalated.
- Hidden cost. Every hop is another model call. Cheap routing (rules first, small model second) keeps this in check; see agent cost control.
How Mai AI uses it
Routing is the backbone of most systems I design. The AI chief of staff is the clearest example: a chief of staff takes each request and hands it to one of six director agents, each responsible for its own area of the business. Every action is checked against what that agent is allowed to do, and anything that matters needs a signed approval token.
In a Mai AI system, I draw the specialists from the client’s real organisation chart rather than from a template, so each agent maps to a person or team who can own its instructions and review its work. Each specialist reaches business systems only through the tool layer, with permissions scoped to its area, and every routing decision is logged.
I also keep the router modest. It classifies, adds context and hands over; it does not take actions of its own. If you want to see what the specialists would be in your business, you can sketch it in the designer for a custom AI operations system.
Questions and answers
What is a router agent?
A router agent reads an incoming request, decides what kind of work it is, and passes it to the agent or process that handles that kind of work. It usually does little of the work itself.
Why use multiple specialist AI agents instead of one?
Each specialist gets focused instructions and only the tools and data its area needs. That makes behaviour easier to test and limits what any single agent can reach if something goes wrong.
What is an agent handoff?
A handoff is when one agent passes control of a conversation or task to another. In frameworks such as the OpenAI Agents SDK, the model sees each possible handoff as a tool it can call.
Can routing send simple questions to cheaper AI models?
Yes. Routing can choose a model as well as an agent, sending easy requests to a smaller, cheaper model and harder ones to a more capable one.
Harnesses known for this
- Harness profileOpenAI Agents SDK
The OpenAI Agents SDK is an open-source library, in Python and TypeScript, for building agents that use tools, hand work to each other and pause for approval. It suits teams that want a light agent loop with tracing built in, rather than a full orchestration platform.
- Harness profileOpenClaw
OpenClaw is an open-source, self-hosted AI assistant that runs on your own machine and answers you in the chat apps you already use, such as WhatsApp, Slack, Teams and Telegram. It is best for one person or a small trusted team who want an always-on assistant with scheduled jobs, skills and tools under their own control.
- Harness profileLangGraph
LangGraph is an open-source framework from LangChain for building stateful agents as graphs of steps, with saved checkpoints so a run can pause, wait for a person and resume. It is best for long-running, multi-step work where you want explicit control over every path.
- Harness profileCrewAI
CrewAI is an open-source Python framework for building teams of agents, each with a role, a goal and tools, that work through tasks together. It is best for work that splits naturally into roles, wrapped in event-driven Flows that keep the overall process under control.
- Harness profileMicrosoft Agent Framework
Microsoft Agent Framework is Microsoft’s open-source framework for agents and multi-agent workflows in .NET and Python, and the direct successor to both AutoGen and Semantic Kernel. It is best for teams on Azure or .NET that want typed, checkpointed workflows with approvals and OpenTelemetry tracing.
Related entries
- PatternSub-agents and parallel work
Sub-agents are helper agents that a lead agent starts for one well-defined piece of a larger job, each with its own fresh context, instructions and tools. Several can run at once and report back short findings, which makes broad tasks faster, at the price of more model usage.
- PatternThe tool layer and MCP
AI agents reach real systems through tools: named functions such as "find invoice" that the model asks to run, while ordinary code does the actual work. MCP, the Model Context Protocol, is an open standard for offering those tools and data to any AI application in the same way.
- PatternHuman approval gates
Human approval gates sort every action an AI agent can take into three levels: draft only, needs a person to approve, or allowed to run automatically. The agent prepares, a person signs off anything that matters, and the system refuses any action that does not match what was approved.
- PatternAgent cost control
You control AI agent costs by sending routine work to small, cheap models and saving large ones for hard steps, caching repeated prompt content, and putting hard budgets and caps on every key, agent and job. Clients can then pay providers directly on their own keys or use a managed allowance.
Sources
- Building effective agents (Anthropic engineering)anthropic.com
- Handoffs (OpenAI Agents SDK documentation)openai.github.io
- LLM06:2025 Excessive Agency (OWASP GenAI Security Project)genai.owasp.org