When a company is considering deploying an AI agent, the first question tends to be what development will cost. That's understandable — it's a one-off item, it can be planned and approved. The question asked less often is what the agent will cost every month afterwards, once it's running.
The difference from conventional software is significant. An ordinary web application has a running cost that is largely fixed — hosting costs roughly the same whether twenty or thirty people use it. An AI agent has a cost that changes with every message, and it grows in a way most people don't expect.
What the bill is made of
An agent's running cost has three components, and each behaves differently.
Language model calls. This is typically the largest and most volatile item. It's billed by token count — the chunks of text that go into the model and come out of it. Input is usually cheaper than output, but there tends to be an order of magnitude more input.
Infrastructure. Server, database, job queue, storage. This part behaves like any application and is predictable.
Vector database and search, if the agent works with company documentation. The cost grows with the volume of documents and how often they're reindexed, not with the number of questions. We covered how that kind of solution works in our article on RAG and company documentation.
What actually drives the bill up
The intuitive answer is "the number of questions". In practice the main factor is somewhere else: the length of the context sent to the model on every call.
An agent doesn't remember a conversation by itself. For it to know what was said, the history is sent along with each new question. If the whole history is sent, every subsequent message in the same conversation costs more than the last — you pay for the same sentences again on every call. A long conversation therefore costs disproportionately more than several short ones.
The same applies to context from documentation. If the agent attaches the twenty longest retrieved passages to every question instead of the three relevant ones, it pays for eighteen unnecessary ones every time.
| Factor | Effect on the bill | Can it be influenced |
|---|---|---|
| Number of conversations | direct, linear | hardly — it's success, not a problem |
| Length of a single conversation | more than linear | yes — trimming and summarising history |
| Amount of attached context | direct | yes — better retrieval, fewer passages |
| Choice of model | multiplicative | yes — but watch the quality |
| Repeated identical questions | direct | yes — caching |
What can actually be done
The first lever most people reach for is a cheaper model. It's a legitimate one, but not the first — a drop in answer quality shows up as escalations, and those cost people's time, which tends to be more expensive than the tokens saved. We cover how to approach that choice in our article on choosing a language model.
It's more effective to start with what gets sent to the model in the first place:
- Don't send the whole history. Older parts of a conversation can be replaced with a short summary. What matters to the agent is what the client wants, not a verbatim transcript of the greetings.
- Send fewer but more relevant passages. Better retrieval means a shorter context and a more accurate answer at the same time — the one place where quality and cost move in the same direction.
- Cache repeated questions. In customer support a large share of enquiries are near-identical. An answer that has already been produced doesn't need generating again.
- Don't send the model what the model isn't needed for. "What are your opening hours" doesn't need a language model. A simple filter in front of the agent resolves a share of enquiries at zero cost.
- Pick the model per task, not one for everything. Sorting and classification can run on a smaller model; wording an answer to a customer deserves a better one.
Monitoring: what to measure from day one
Without measurement you can't optimise, and a mistake surfaces only on the invoice. The minimum worth tracking:
- Token consumption per day, split into input and output.
- Average context length per call — if it grows over time, something is accumulating.
- Cost per conversation, not just the monthly total. This is the number you can compare against the value a conversation brings.
- Share of escalations to a human. If it jumps after switching to a cheaper model, the saving was merely moved elsewhere.
- A budget cap and an alert. A threshold that triggers a warning — and, if badly exceeded, stops the agent.
How to factor it into the decision
When comparing against a manual approach it pays to count total cost, not just development: the one-off implementation, monthly operation, and the people's time the agent still requires — oversight, expanding the knowledge base, handling escalations. We go at this methodically in our article on measuring the ROI of deploying an AI agent.
Concrete figures differ between projects enough that a flat number would be misleading — they depend on communication volume, conversation length, the amount of documentation and the model chosen. That's why an estimate only makes sense against a specific brief, verified by measurement during a pilot.
Summary
An AI agent's running cost is a variable item driven mainly by the length of the context being sent, not by the number of questions. The cheapest optimisations are the ones that shorten context and filter out enquiries the model was never needed for — not switching to a cheaper model. And from day one you need consumption measured, with a cap and an alert.
If you're weighing up a deployment and want to work through what the running cost would look like in your case, get in touch for a no-obligation consultation or take a look at our AI and automation solutions.