Fundamentals
Which model should you use? Wrong question.
Model choice is a measurement, not a belief. How to build so the model stays swappable, and when switching is actually worth the effort.
One of the most common questions we get: “which model do you work with?”
The answer nobody enjoys: it depends, and we measure it.
Why the model isn’t the deciding factor
Look at the stack at fifty companies and you’ll find roughly the same thing. The same frontier models, the same editors, the same agent frameworks. If everyone has access to the same thing, access isn’t an advantage.
The advantage comes from who can embed it into their own processes. And most of that work — process discovery, exception handling, integration, logging — is entirely independent of which model runs underneath.
So model choice isn’t a strategic decision here. It’s a measurement.
How to build for swappability
Nothing mysterious. Four things.
1. Model calls live in one place. A single layer that knows which provider a request goes to. The rest of the application doesn’t.
2. Structured input and output. Schema-defined JSON, not free text you’re trying to unpick with a regex. Then switching doesn’t rewrite the calling code.
3. An eval set on your own data. This is the crux. Without it, switching is a belief: “they say it’s better.” With it, it’s half a day: run it, compare.
4. Cost and latency measured per run. Without this you can’t decide, because you don’t know what you’d gain.
With those four, a model switch is one config line and a measurement.
But if you’re just starting: pick one
This sounds contradictory. It isn’t.
As a company we build model-agnostic, because that’s an architectural decision. But if you’re learning this, the worst strategy is knowing a little of everything.
Pick one ecosystem and learn it deeply: its agent framework, its tool calling, its context handling, its failure modes, its cost model. Most of that knowledge transfers. The ability to compare, though, only develops once you have something to compare against — which means knowing the first one very well.
When switching is worth it
Four cases:
The measurement is better and the cost isn’t worse. The trivial case.
Same measurement, cheaper. More common than you’d think, especially on sub-tasks. A well-bounded classification step doesn’t need a frontier model.
Latency matters. In an interactive flow, a faster and slightly weaker model gives a better overall experience.
There’s a compliance reason. Data residency, a processing agreement, a contractual clause. This sometimes overrides every measurement.
When it isn’t worth switching: because something new and exciting appeared. Switching has its own cost — remeasuring, retuning, new failure modes.
The mixed setup
In practice most of our live systems run several models at once. A typical split:
| Step | What runs | Why |
|---|---|---|
| Document classification | smaller, cheap model | narrow task, many calls |
| Extraction | stronger model | errors are expensive here |
| Schema validation | plain code | no model needed |
| Final coding | stronger model | requires judgement |
| Summary text | mid-tier model | good enough, cheaper |
That split didn’t come from theory. It came from measuring, at every step, what we’d lose by going down a tier.
Related: token maxing and evals.
Questions on this topic
If I'm starting out, which ecosystem should I pick?
One, and learn it properly. In the early months your own learning curve is the bottleneck, not the model's capability. Once you know one stack deeply, the second takes weeks.
Is it worth using a smaller model for sub-tasks?
Often yes, and it's the best-returning optimisation available. A well-bounded classification step gets the same result from a smaller model at a fraction of the price — but only switch after checking it on the eval set.
What does this look like at your company?
If this problem sounds familiar, let's start with one process. Tell us which department burns the most manual hours — we'll come back with a concrete proposal.