Analysis from Cerebras of the cost-performance tradeoff of reasoning (“test-time compute”) in LLMs. The headline numbers are stark: reasoning costs 6x more tokens for a 10-20% accuracy improvement, and roughly half of real-world agent prompts don’t benefit from it at all.
The Cost Multiplier
The article opens with concrete figures from real usage:
- 6x more tokens for technical prompts with reasoning enabled
- 7-11x longer completion times
- 10-20% improvement in benchmark/coding performance
For models like Qwen3.6-27B, 87.5%+ of generated tokens are reasoning overhead. Disabling reasoning therefore cuts the bill by ~85% and significantly reduces KV-cache compaction in memory-constrained environments, letting agents run longer before hitting context limits.
Where Reasoning Helps vs Hurts
Reasoning (interleaved thinking between tool calls, Socratic self-questioning, plan reevaluation) is valuable for:
- Complex single-shot challenges (puzzles, math, logic)
- Hitting benchmark targets
- Tasks requiring precise multi-step reasoning chains
But an analysis of 1000+ AI sessions with Codex, Claude Code, Droid, and Pi agents showed roughly half of prompts were simple: “find and open x file”, “check my email & calendar”, “locate processes on my machine”, “clone and analyse this repo”, “download and configure x resource”. These see zero benefit from reasoning but pay the full token cost.
Reasoning as a Speed Toggle
The key insight: reasoning should be treated as a speed/cost control, not a default. Anthropic and OpenAI charge 2x for 1.5-2.5x speedups on their models - but the article points out you can go 7x faster just by turning off reasoning and accepting ~20% less capability on hard tasks.
For time-sensitive work (file fetching, incident response, incremental updates, AI-as-system-component), paying the reasoning toll makes no sense.
Types of Reasoning
Three patterns identified:
- Interleaved thinking - current standard; model thinks between tool calls, weighing context against next action
- Adaptive reasoning - model decides how much to reason autonomously
- Configurable reasoning - user selects level (low/medium/high)
Adaptive and configurable are the more economically sensible approaches - they let cheap, simple queries skip the overhead.
Lessons worth keeping
- ~50% of real agent prompts are simple enough that reasoning is pure waste - cost-aware prompting matters
- Disabling reasoning can cut AI bills by 85% and extend agent runtime before compaction
- The 10-20% accuracy boost from reasoning comes at 6x token cost and 7-11x time cost
- Configurable reasoning (user-selectable level) is the pragmatic middle ground
- Reasoning type matters: interleaved (current default) vs adaptive vs configurable - they have very different cost profiles
Sources
- Cerebras blog
- Full text: economics-of-ai-reasoning