Whenever an AI agent does something stupid, the default reaction is usually:
Upgrade the model.
Reasoning isn't good enough? Use the bigger model. Coding sucks? Switch to Pro. Agent keeps failing? Clearly, we need more intelligence.
The API bill?
That's a problem for Future You.
But recent research suggests an uncomfortable possibility:
Maybe the model isn't the first thing you should upgrade.
A bigger engine doesn't help if the steering is broken
Imagine buying a 500-horsepower tractor.
Amazing engine. Latest technology. Number one on TractorBench.
There is just one small problem:
The steering is misaligned, the plow is mounted incorrectly, the driver can't see the field, and nobody tells him when he's gone off course.
What's the obvious solution?
Buy the 700-horsepower version.
Sounds stupid.
Yet we do roughly the same thing with AI.
An agent isn't just a model.
Around the model sits a whole system: context, tools, retrieval, memory, state, permissions, validation, retries, recovery, feedback loops, and orchestration.
That surrounding system is what we increasingly call the harness.
And it matters a lot.
Harness-Bench, a 2026 study covering 106 tasks and 5,194 execution trajectories, found substantial differences in completion rate, efficiency, process quality, and failure behavior depending on the model–harness pairing.
The authors argue that agent capability should be reported at the model–harness configuration level, rather than attributed to the base model alone.
In other words:
The model provides horsepower. The harness decides whether that horsepower turns into useful work.
Sometimes the smaller model actually wins
This is where it gets fun.
The AutoHarness paper looked at a surprisingly dumb failure mode.
In the Kaggle GameArena chess competition data analyzed by the researchers, 78% of Gemini 2.5 Flash losses were attributed to illegal moves.
Not bad strategy.
Not insufficient chain-of-thought.
Illegal moves.
So instead of simply throwing a larger model at the problem, the researchers had Gemini 2.5 Flash synthesize a code harness that constrained its actions.
The resulting harness prevented illegal moves across 145 TextArena games and allowed the smaller Gemini 2.5 Flash to outperform Gemini 2.5 Pro in their experiments.
Basically:
A 50-horsepower tractor going down the correct row can still beat a 200-horsepower tractor plowing your neighbor's field.
And this gets interesting when money enters the chat
Better Harnesses, Smaller Models studied whether cheaper small language models could become competitive when their harnesses were adapted to their weaknesses.
Across seven business-oriented tasks and three SLM families, optimized harnesses significantly improved 16 of 21 model–task pairs.
Seven pairs closed the performance gap with the larger LLM.
In the best case, a smaller model recovered 89.7% of the larger model's performance at only 4% of the cost.
The most interesting idea in the paper isn't even that 4% number.
It's this:
Some of the difficulty of a task can be moved out of the model and into the harness.
Instead of making the model remember everything, give it better memory.
Instead of making it guess how an API works, give it a better tool interface.
Instead of dumping an entire database into context and praying, retrieve the right information at the right time.
Instead of letting an agent make a mistake and confidently continue for another twelve steps, give it validation and recovery loops.
This idea isn't entirely new either. SWE-agent showed back in 2024 that designing an appropriate Agent-Computer Interface could significantly change how effectively language-model agents navigate repositories, edit files, and execute tests.
OpenAI reported something similar from building software with Codex. Early progress was slower than expected not simply because the model lacked capability, but because the environment lacked the tools, abstractions, and structure the agent needed. The engineering work increasingly became about designing environments, constraints, and feedback loops around the model.
So the next time your agent does something stupid...
Before saying:
“We need a better model.”
Ask a few other questions first.
Is retrieval giving it the right information?
Are the tools easy for the model to understand?
Is the context useful, or is it mostly garbage?
Can the agent tell when an action failed?
Can it recover when something goes wrong?
Do you even have an eval that tells you where it's failing?
Better models absolutely matter.
But a better model does not automatically create a better system.
Before buying the 700-horsepower tractor, maybe check whether...
the steering wheel is actually connected to the wheels.
References
References
- Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows — 2026.
- AutoHarness: improving LLM agents by automatically synthesizing a code harness — 2026.
- Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation — 2026.
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — 2024.