AI agent development
Most agent projects die between the impressive prototype and the first thousand real cases. We build the unglamorous layer that closes that gap — grounded retrieval, validated tool calls, evaluation suites and escalation paths — so the agent still works in month six.
What is ai agent development?
AI agent development is the engineering work of turning a language model into a system that completes a business process on its own — reading from your systems, deciding what to do, taking action through tools, and escalating to a human when it is not confident. The hard part is not the model. It is the retrieval, the tool contracts, the guardrails, the evaluation suite and the observability that keep the agent correct once it is handling thousands of real cases a day rather than five in a demo.
Why teams call us
Almost every engagement starts from one of these. If none of them sound familiar, you probably do not need us yet.
A prototype tested on a dozen hand-picked cases hides everything that matters. Real inputs are messier, real users ask things the prompt never anticipated, and the long tail of edge cases is where the process actually spends its time. Without an evaluation set drawn from your own historical cases, nobody discovers this until the agent is live.
Typically 2-3 months of rework, plus the internal credibility cost of a visible failure.
Teams without an evaluation harness end up tuning prompts by vibe. Someone changes a wording, it looks better on the three examples they checked, and a regression ships silently. Six weeks later the agent is measurably worse than at launch and there is no record of which change caused it.
Quality drifts downward invisibly until a customer or auditor finds it.
Agents are given broad credentials early because it is faster, and the permissions are never narrowed. The result is an autonomous system with write access to production data, no spend ceiling, no approval step on high-impact actions, and no audit trail that would let you reconstruct what it did.
One incident is usually enough to get the whole programme paused.
Routing every step through a frontier model — including classification, extraction and routine formatting — makes unit economics collapse as volume grows. The bill is fine in pilot and indefensible at scale, and by then the architecture assumes one model everywhere.
Commonly 4-10x more inference spend than the same workload needs.
Scope
We start by deciding what should not be an agent. Deterministic steps stay deterministic; the model is used only where judgement is genuinely required. That single decision is the difference between a system you can debug and one you cannot.
Agents are only as useful as the actions they can take. We build typed tool contracts against your CRM, ERP, ticketing, database and internal APIs, with validation at the boundary so a malformed model output can never reach a system of record.
An agent reasoning over stale or irrelevant context will confidently do the wrong thing. We build the retrieval layer — chunking, hybrid search, reranking, freshness handling — and make every answer traceable to the source document it came from.
We classify every action by blast radius and design the approval path accordingly. Reversible, low-impact actions run autonomously; anything that moves money, touches a customer record or cannot be undone gets a human checkpoint.
We build an evaluation set from your real historical cases, including the ones that went wrong, and wire it into CI. Every prompt, model or retrieval change is scored against it before it ships.
Once live, you need to see quality, latency and cost per action — not just error logs. We instrument traces end to end and route each step to the cheapest model that passes evaluation for it.
How it runs
Fixed checkpoints with a named deliverable at each one, so you can stop the engagement at any stage and still own something useful.
Two weeks, fixed price. We map the process end to end, identify which steps genuinely need a model, assemble the first version of the evaluation set from your historical data, and produce an architecture with a cost and latency model. If the honest conclusion is that this should not be an agent, that is what the document says.
We build the narrowest complete path through the process — one real case type, running against real systems, with retrieval, tool calls, guardrails and logging in place. It goes to a small internal group so the failure modes surface while they are cheap to fix.
The evaluation suite is expanded to cover the long tail, and we work the failure modes it exposes: retrieval gaps, ambiguous instructions, tool errors, adversarial inputs. This is the phase most projects skip, and skipping it is why they fail.
We ship behind a feature flag, ramp traffic in stages, and keep a human review queue oversized at first so errors are caught by us rather than by your customers. Ramp decisions are made on evaluation and incident data.
Either your team takes it — with documentation, a recorded architecture walkthrough and paired sessions on the evaluation tooling — or we operate it on a retainer. We will tell you which is cheaper for you over two years.
Results
Building something impressive with a language model is now easy. A competent engineer can wire up a tool-calling loop over a weekend and produce a demonstration that makes a leadership team genuinely excited. That is the trap: the prototype is the cheap 20% of the work, and it creates the impression that the remaining 80% is a formality.
It is not. The prototype succeeds because it runs on a handful of clean, chosen examples. Production fails because real inputs are ambiguous, the process has a long tail of exceptions nobody documented, source systems return unexpected nulls, and users ask for things outside the intended scope on their first day.
The work that closes that gap is unglamorous and mostly has nothing to do with prompts. It is retrieval quality. It is validating model output against a schema before anything acts on it. It is deciding which actions require human approval. It is building an evaluation set so a change can be measured rather than guessed at. None of that demos well, and all of it determines whether the agent is still running in six months.
Strip away the marketing and an agent is a loop with four parts:
Everything that distinguishes a production agent from a prototype lives in the constraints around that loop. How many iterations before it gives up. What it is permitted to call. What happens when a tool errors. What confidence level is required before it acts unsupervised. How a human takes over mid-task. Whether you can reconstruct, three months later, exactly why it made a particular decision.
On the agent builds we have shipped, the effort distribution is consistently lopsided:
| Area | Share of effort | Why it dominates |
|---|---|---|
| Retrieval and data plumbing | ~30% | Agents fail on context quality far more often than on reasoning quality. |
| Tool contracts and integration | ~25% | Every system has undocumented behaviour, and the agent finds all of it. |
| Evaluation and testing | ~20% | This is the only thing that makes quality measurable rather than anecdotal. |
| Guardrails and permissions | ~15% | Blast radius has to be designed, not discovered after an incident. |
| Prompting and model selection | ~10% | Real, but the smallest slice — and the easiest to change later. |
Teams that expect the reverse distribution — most effort on prompts — are the ones that get surprised.
If we could enforce only one practice, it would be this: before writing agent logic, assemble a set of real historical cases with known correct outcomes, including the awkward ones and the ones a human got wrong.
That set becomes the thing you optimise against. It turns “this prompt feels better” into a number. It catches the regression introduced by a model version change. It gives you a defensible answer when someone asks how accurate the system is. And it is the only mechanism we know of that stops slow quality decay over months of small changes.
Building it takes real work — usually a week of pulling records and adjudicating correct answers with someone who knows the process. Teams resist it because it feels like a detour. It is the shortest path.
A recurring mistake is treating safety as a prompting problem — writing “never issue a refund above $500” into the system prompt and considering it handled. Instructions are the weakest available control. They are probabilistic, and a sufficiently unusual input will route around them.
The controls that hold are structural, and they sit outside the model:
The most common unit-economics mistake is running every step through a frontier model. Agent workflows are full of routine steps — classifying an intent, extracting a field, formatting an output — where a small model performs identically at a fraction of the price.
We benchmark step by step and route each one to the cheapest model that passes evaluation for that step, keeping frontier models for the genuinely judgement-heavy parts. On high-volume workloads this routinely cuts inference spend by four to eight times with no measurable quality change. Doing it later is harder, because by then the architecture assumes one model everywhere.
We start with a fixed-price two-week scoping sprint, and its output is yours whether or not you continue: architecture, evaluation plan, cost model, estimate. We do that because a scope you cannot take to another vendor is not a scope, it is a lock-in mechanism.
Sometimes that document concludes you should not build an agent — that the process needs fixing first, or that a deterministic workflow would be cheaper and more reliable. We would rather write that in week two than bill you for twelve weeks of building the wrong thing.
Next Step
A 30-minute technical call with an engineer who has shipped this before — not a sales qualification round. You leave with a feasibility read, a rough shape for the build, and an honest answer about whether it is worth doing at all.