Agentic AI Architecture: Patterns, Controls and What It Takes to Run Agents in Production

Agentic AI architecture is the design of systems in which a language model chooses and executes steps toward a goal, usually by calling tools in a loop, together with the controls that bound it: which tools it can reach, whose identity it acts under, how its state survives failures, where humans approve, and how its behavior is observed and evaluated. The model supplies the reasoning. The architecture decides whether that reasoning is allowed to touch your systems, and how far.

This guide covers the core agentic AI architecture patterns, when each fits, and the cross-cutting controls that every production agent needs regardless of pattern.

Start with the least agency that works

“Agent” covers a wide range of designs. It helps to place any proposal on a spectrum:

1. Workflow with model steps. Code defines the sequence; the model classifies, extracts or drafts at fixed points. Predictable and easy to test.
2. Router. The model chooses which of several predefined paths to take.
3. Single agent with tools. The model decides which tools to call, in what order, until it judges the task done.
4. Planner-executor. The model writes a plan, then executes and revises it.
5. Multi-agent. Several agents with different roles coordinate, usually under a supervisor.

Each step to the right adds flexibility and adds failure modes. Many problems pitched as agents are better solved as workflows with model steps. A sound agentic AI architecture uses the least agency that meets the requirement, and saves open-ended autonomy for tasks where the path genuinely cannot be known in advance.

Pattern 1: Single agent with tools

One model, one set of instructions, a list of tools with typed input schemas, and a loop: the model calls a tool, reads the result, and decides the next step until it returns an answer or hits a limit.

This is the workhorse pattern and should be the default. Design notes:

• Keep the tool list short and distinct. Tools with overlapping purposes confuse selection. Write tool descriptions as carefully as you would write an API contract.
• Make tools narrow. “Look up order status by order ID” is safer and easier to evaluate than “run a database query.”
• Return structured, compact results. Large raw payloads waste context and bury the relevant fields.
• Bound the loop. Set limits on steps, elapsed time and spend, and define what happens when a limit is hit.
• Validate outputs. Check tool arguments against schemas before execution and final outputs before they leave the system.

Pattern 2: Planner-executor

The model first produces an explicit plan (a list of steps with expected inputs and outputs), then executes the steps, sometimes with a separate executor, and re-plans when a step fails or returns something unexpected.

This helps on longer tasks such as research, multi-system reconciliations or document preparation, because the plan becomes an inspectable artifact. You can log it, evaluate it, and show it to a person for approval before anything runs. The cost is extra model calls and the risk of rigid plans that do not adapt, so re-planning rules matter as much as the plan itself.

Pattern 3: Supervisor and multi-agent

A supervisor agent receives the task, delegates parts of it to specialist agents (each with its own instructions, tools and permissions), and assembles the result. Frameworks such as LangGraph provide building blocks for these graphs.

The legitimate reasons to split into multiple agents are separation of permissions (the agent that reads HR data should not also send external email), distinct instructions that conflict when combined, and tool sets too large for one agent to use reliably. “It mirrors our org chart” is not a good reason.

Multi-agent designs multiply cost and latency and make failures harder to trace, because errors propagate through handoffs. If you use this pattern, define explicit handoff contracts (what each agent receives and must return), trace every handoff, and evaluate each agent separately as well as end to end.

Pattern 4: Human-in-the-loop checkpoints

Human oversight is a pattern you design in, not a disclaimer you add later. Place checkpoints where the cost of an error is high:

• Before irreversible or externally visible actions: sending a customer email, filing a document, making a payment, changing access rights, deleting data.
• When a policy threshold is crossed: a refund above a set amount, a contract clause outside standard terms.
• When the agent’s confidence is low or its inputs conflict.

At each checkpoint, show the person what the agent intends to do and why, and let them approve, edit or reject. Capture the decision as training and evaluation data. Because a person may take hours to respond, checkpoints depend on durable execution, covered below.

Choosing a pattern

Pattern	Best for	Main risks	Key control
Workflow with model steps	Known processes with some judgment steps	Brittle when inputs vary widely	Schema validation at each step
Single agent with tools	Most interactive and task-completion use cases	Wrong tool choice, runaway loops	Narrow tools, step and cost limits
Planner-executor	Longer multi-step tasks	Rigid or poor plans	Plan review and re-planning rules
Supervisor and multi-agent	Tasks needing separated permissions or instructions	Cost, latency, opaque failures	Handoff contracts, per-agent evaluation
Human checkpoints (any pattern)	High-impact or regulated actions	Approval fatigue, rubber-stamping	Clear context, sensible thresholds

Tool access through MCP

The Model Context Protocol (MCP), introduced by Anthropic in November 2024, is an open protocol for connecting AI applications to tools and data. An MCP server exposes tools, resources and prompts in a standard way; any MCP-capable client can use them. For enterprises the attraction is consistency: build a governed connector to a system once and reuse it across assistants and agents, instead of writing bespoke integrations per application.

Consistency only helps if the servers are governed. Treat MCP servers like any other integration:

• Maintain an allowlist and registry of approved servers, with owners and data classifications.
• Require authenticated connections for remote servers, with scoped, short-lived credentials.
• Review tool descriptions as well as code, since descriptions are read by the model and can carry instructions.
• Log every tool call with the calling agent, user and arguments.
• Treat all tool outputs as untrusted input to the model.

Our MCP for the enterprise guide goes deeper on deployment and governance.

Identity and permissions

The most important question in agent design is: whose authority is this action taken under?

There are two basic models. In delegated access, the agent acts on behalf of a specific user and should never exceed that user’s permissions; the user’s identity is passed through to each tool, usually as a scoped token. In service identity, the agent acts as itself for background tasks; it needs its own identity with narrowly defined rights, owned by a named team.

Principles that hold in both models:

• Least privilege per tool, not per agent. A read tool and a write tool should use different scopes.
• No ambient credentials. Do not give the agent a broad API key in its environment. Issue short-lived tokens per task.
• Authorize at the tool, not in the prompt. The tool checks whether this identity may perform this action on this resource. Instructions in the prompt are not access control.
• Audit on-behalf-of chains. Logs should show the user, the agent, the tool and the resource for every action.

The OWASP Top 10 for LLM Applications names excessive agency and prompt injection as major risks, and they compound in agents. Be especially careful with any agent that combines access to private data, exposure to untrusted content (email, web pages, tickets, documents), and the ability to send data outward. Removing one of those three capabilities, or putting a human checkpoint in front of outbound actions, sharply reduces exposure. See our LLM security threat model for a fuller treatment.

Durable execution and state

Demos run in a single process for a few seconds. Production agents run for minutes or days, call slow external systems, wait for human approval, and hit timeouts, rate limits and crashes. Without durable execution, a failure halfway through a task either loses the work or, worse, repeats side effects when retried.

The architecture needs:

• Persisted state and checkpoints after each step, so a task can resume where it stopped. Durable workflow engines such as Temporal, AWS Step Functions or Azure Durable Functions provide this, and agent frameworks such as LangGraph offer checkpointing for graph state.
• Idempotent tools. Every side-effecting tool should accept an idempotency key so a retried call does not send the email or create the ticket twice.
• Explicit timeouts and retries per tool, with backoff, and a defined dead-letter path for tasks that cannot complete.
• Budgets for steps, tokens and wall-clock time, enforced outside the model.
• Versioning. Record the model, prompt and tool versions each task ran with, so in-flight tasks are not broken by a deployment and results can be reproduced.

Observability and evaluation

Agent quality is a property of the whole trajectory, not a single response. Trace every step: prompts, model outputs, tool calls and arguments, tool results, approvals, cost and latency. Evaluate on task success rates against a representative task set, on trajectory quality (unnecessary steps, wrong tools, policy violations), and on cost per completed task. Test new versions in a sandbox with mocked or non-production tools before they touch real systems. Route all model calls through your LLM gateway so cost and usage are attributed per agent.

Production readiness checklist

1. The design uses the least agency that meets the requirement.
2. Every tool is narrow, typed, owned, and authorizes the caller itself.
3. Identity model (delegated or service) is explicit, with least-privilege scopes.
4. High-impact actions sit behind human checkpoints with clear context.
5. State is durable, tools are idempotent, and budgets are enforced outside the model.
6. Untrusted content is isolated from high-privilege tools and outbound channels.
7. Full traces are captured, and a task-level evaluation suite gates releases.
8. A named owner can pause or roll back the agent quickly.

How we help

We design agent architectures that start narrow and earn autonomy with evidence, and we build the first production agents with your engineers, including tool servers, identity integration, durable execution and evaluation. See our agentic AI architecture service, or contact us to review an agent design before it reaches production.