For leaders
Token maxing: how to burn a year's budget in three months
Handing everything to the model is more expensive, slower and less reliable than code would have been. Where the money goes, and what to do instead.
There was a period when the strategy amounted to “the models are good enough, give them everything”. That’s token maxing.
We know an executive who burned a full year’s budget in three months this way. The sad part wasn’t the number. It was that nothing moved in the business metrics meanwhile.
Why it’s tempting
Because it works — in a demo. Pour everything into the context and the model generally figures out what you want. No process to map, no steps to define, no decisions about what should stay deterministic.
In a week you have something that looks good in a presentation. The invoice arrives two months later.
Where the money goes
The same context on every call. Send fifty pages of policy with every single case and at a thousand cases a day you’re paying for sixty thousand pages a month — the same sixty thousand.
Agents that wander. With no clear stopping condition, an agent solves in fifteen steps what three would have done. That doesn’t show up in operations, only on the bill.
Model calls at steps that don’t need them. “Is this amount over a million?” doesn’t need a model. We see it regularly anyway.
Retries without measurement. If the system tries three times because it can’t tell whether it succeeded, it’s working at triple price.
What works against it
Decide per step whether a model belongs there. This is the FDE’s judgement: in a ten-step process, typically two or three steps need real judgement. The rest is code. That’s usually not a 10–20% saving, it’s an order of magnitude.
Use a smaller model where it’s enough. Classification, extraction, format detection — often a fraction does fine. Five minutes on the eval set settles it.
Cache the constant context. If the same policy goes out on every call, most providers offer discounted caching for it. Typically the largest saving available for the least work.
Give it a stopping condition. Maximum steps, maximum cost per case. On reaching it, route to a human — don’t keep trying.
Measure per case, not per month. If you know a case costs nine cents, you can make a business decision. If you only see the monthly invoice, you can only panic.
What a leader should ask
If someone is running an AI project at your company, these four questions reveal a surprising amount:
- What does one case cost to process? No number means no measurement.
- At how many steps do we call a model, and why those? If the answer is “everywhere”, there’s room to optimise.
- What’s the stopping condition? Without one, cost is unbounded above.
- How much worse would a cheaper model be? No answer means there’s no eval set — and that’s the bigger problem.
The lesson
Token maxing isn’t bad because it’s expensive. It’s bad because it hides the design work. As long as everything goes to the model, it never becomes apparent that you don’t understand the process. Then it falls over in production and you’re standing there with a large bill and still no understanding.
The cheaper system is usually the same as the more reliable system. That isn’t a coincidence.
Related: which model should you use and measuring AI ROI.
Questions on this topic
How do we forecast the monthly cost?
We estimate it during discovery from volume and per-step token usage, then measure it on real data in shadow mode. Before go-live you get a concrete monthly figure, not an order of magnitude.
Should we set budgets per team?
Yes, but a budget alone isn't enough — it needs alerting that fires in time. The budget says how much you may spend; the measurement says whether it was worth it.
What does this look like at your company?
If this problem sounds familiar, let's start with one process. Tell us which department burns the most manual hours — we'll come back with a concrete proposal.