Tool-calling accuracy: Why context is the real AI agent bottleneck

Tool calls are not the problem, but context can create one.

For me, this learning journey started in an everyday code review. I was looking at a PR for one of our chart-building agents, the segment builder, where an engineer had bolted a new capability onto an existing tool instead of creating a separate one.

I argued for a separate tool.

The pushback was the conventional wisdom: too many registered tools degrade quality. The model gets distracted, loses the thread in a long context, or picks the wrong function. We both had a version of best practice on our side, so instead of arguing I devised a test.

What my test actually showed

I stress-tested tool selection as the number of registered tools grew from a handful up to a hundred, across single calls and longer sequential chains.

The short version: on a current frontier model, the "too many tools" worry mostly didn't hold. It stayed reliable well past the point the old rule of thumb says it should break. An older, cheaper model was a different story, erratic as the toolset grew.

I'm not going to turn this into a benchmark you can pick apart on a trial-by-trial basis, because the exact curve isn't the point. The point is that model capability has moved, and a lot of the scaffolding we build to compensate for "dumb" models is quietly solving a problem that's going away.

So where does the hard part go?

If tool accuracy is close to solved, the hard part moves upstream, to whether the model can get the context it needs, whenever it needs it. Sometimes that means loading the context up front.

More often, in an agentic loop, it means the tool itself returns enough alongside the raw data: not just the event, but what the event means. A capable model will pick the right tool. It will not always take the most effective path to an answer, especially when the request is vague or the data is unlabeled.

That's when the real work starts: reasoning about the problem, doing discovery, running fail-fast experiments to move forward without burning cycles. The constraint is no longer whether the model can act. It's whether it can reach the context, on its own, at the moment it needs it.

And when a model doesn't have that context, it doesn't just get the wrong answer, it keeps trying, and every attempt has a token cost. More on that later.

A fresh example

Last week, I was testing agents against a customer's analytics setup, and their custom instrumentation was messy and thinly named.

A simple question, "How many people completed account registration today?", kept tripping the agent up. The event that captured the relevant data was named something like evt_4f2Complete, with no description.

The model had no way to know what that was. What helps is context.

If the agent can see that evt_4f2Complete feeds a metric called "Account registrations," it can infer its meaning and proceed with confidence. Same model, same question. The only thing that changed was whether the data around the event told the model enough to understand it.

This is a cost problem, not just an accuracy one

The token cost is what the people who approve the cloud bill should care about.

In one test, a question that lacked the right context sent the model into a loop. It burned millions of tokens and took far longer than any user would sit through, and still gave up with a barely accurate answer. The same class of question, with the context in place, came back accurate on a small fraction of that. A model that can't find what it needs doesn't fail fast. It keeps trying, and you pay for every attempt.

Two takeaways, depending on your seat

For engineers: treat tools like Unix utilities. Each should do one thing well. On a current model, stop optimizing for tool count and start optimizing for the clarity of the tool and the data it reaches.

For the people who own the cloud spend: a pricier model can be the cheaper one. We'd been looking at cheaper open-source models to cut costs. But if a cheaper per-token model takes twice the tokens to reach the same answer, it costs more, and it's slower to the user. Per-token price is not the same as cost-to-answer.

The bottom line

We spend a lot of engineering effort making models smarter about acting when the models are already good enough to act. The leverage now is upstream, in whether the data an agent reasons over is complete and self-describing enough to be understood without a human to translate it.

Capability is getting cheap. Context is the frontier. That's where I'm spending my time.