The anatomy of a production AI agent
Knowledge, tools, permissions, a provider abstraction, evaluation and observability: the parts that turn an AI demo into a production AI agent a business can rely on.
A demo agent needs a model and a prompt. A production agent needs everything around them. These are the parts we design before we call an agent ready for real users, and roughly the order we build them in.
- User request
- Guardrails
- Retrieve knowledge
- Model plans
- Tool calls with checks
- Response or handoff
- Logs and evaluation
1. A knowledge layer it can trust
Company facts come from approved, versioned sources with metadata for freshness and visibility. Retrieval pulls the relevant passages, and the agent says it does not know when the answer is not there. Draft or private content never enters the index unless access is explicitly authorised, and the agent only retrieves what the current user is allowed to see.
- A single approved source for each kind of fact
- Metadata for owner, last updated and audience
- Retrieval filtered by the user's permissions
- A clear 'I don't know' when nothing relevant is found
- Citations so people can check where an answer came from
2. Tools with schemas and permissions
Every action is a typed tool with validated arguments and a permission level. Read actions run freely. Writes and external actions require confirmation, and high-impact actions stay with people. The application, not the model, executes every call.
3. A provider abstraction
Routes and components talk to an adapter, not a vendor SDK. That makes it possible to change models, route tasks by cost and quality, and isolate an outage without redesigning the product. It also keeps prompts, tools and evaluations portable across providers.
4. Guardrails at the edges
Inputs are checked for size, rate and obvious abuse before they reach the model. Outputs are checked before they reach the user or a system: structured outputs are validated against a schema, and responses in sensitive domains follow defined boundaries such as declining legal, medical or financial advice.
5. Evaluation before and after launch
| Test set | What a pass looks like |
|---|---|
| Factual questions with known answers | Correct, grounded and cited |
| Questions the agent cannot answer | A clear 'I don't know' with a route to a person |
| Prompt injection in documents and messages | Instructions in content are ignored |
| Tool failures and invalid arguments | Graceful recovery, no repeated loops |
| Out-of-scope requests | Declined politely with an alternative |
The evaluation set is a living asset. Every real failure found in production becomes a new test case, so the same mistake cannot quietly return after the next change.
6. Observability
Log tool calls, failures, latency and cost per session, without storing sensitive content by default. Measure containment, handoff rate and useful-answer feedback. Longer conversations are not automatically better ones, and a rising handoff rate is often the earliest sign something has changed.
How the parts fit together
- One prompt, one model, one happy path
- Answers from the model's general knowledge
- Actions run whenever the model asks
- Judged by a good-looking first conversation
- Guardrails, retrieval, tools and handoffs
- Answers from approved, cited sources
- Actions validated, permissioned and logged
- Judged by evaluation results and live metrics
If the agent feels unreliable, it damages the very proposition it was built to demonstrate.
Frequently asked questions
What is retrieval-augmented generation (RAG)?
RAG is a pattern where the system retrieves relevant, approved documents and gives them to the model as context, so answers are grounded in your own information rather than the model's general knowledge.
What is the difference between an AI demo and a production agent?
A demo shows that something is possible on a good day. A production agent handles bad inputs, unknown questions, tool failures and attacks predictably, is measured continuously, and can be changed safely.
How do you test an AI agent before launch?
With an evaluation set covering known questions, questions it should decline, prompt-injection attempts, tool failures and out-of-scope requests, run before launch and again whenever the model, prompts or tools change.
Why use a provider abstraction for AI models?
So the product depends on an interface you control rather than one vendor's SDK. It lets you change models, route tasks by cost and quality, and survive an outage without redesigning the product.
Get the next note by email
One email a month at most. Practical, from real work. Unsubscribe in one click.