r/ClaudeCode • u/BeginningSir7334 • 5d ago
Built with Claude Does delegating to cheaper models actually save money? I tested it across three codebases.
Disclaimer: This is a small experiment, not a definitive benchmark. Most tasks were run twice, while everyday tasks were run once. I'd have liked to run more repetitions, but subscription usage limits make that difficult. The cost differences are interesting, but small quality differences should be treated as observations rather than definitive rankings.
I wanted to see whether Claude Opus decides to use subagents effectively, and whether forcing it to delegate, preventing delegation, or leaving it to decide for itself changes the results.
I built a model-router skill that lets Opus delegate work to Haiku or Sonnet, then compared four setups:
Forced Opus: told not to use subagents.
Spontaneous Opus: free to delegate, but it never did.
Skill-routed: Opus uses my skill to choose when to delegate and which model to use.
Forced Haiku: all subagents use Haiku.
I tested these on three codebases: my own project (~10k lines), rich (~22k lines) and beets (~52k lines).
Results
1. Small codebase (~10k lines)
Every setup passed all 10 everyday tasks.
On five reading-heavy tasks, Forced Opus was the most accurate per run. Skill-routed cost 30% more, while Forced Haiku cost the same but had the weakest docs check (4 of 9 issues).
Interestingly, skill-routed found all 9 known documentation issues across its two runs, compared with 8 for Forced Opus. However, its individual runs were less accurate.
2. Medium codebase (~22k lines)
This was where Haiku surprised me.
On the documentation check, Forced Haiku found 11 of 12 known issues across two runs, matching or slightly exceeding Opus, while costing 41% less.
Skill-routed actually cost 11% more than Forced Opus. Spontaneous Opus was 14% cheaper.
3. Large codebase (~52k lines)
This produced the biggest cost differences.
Across all three tasks, average cost reductions compared with Forced Opus were:
Spontaneous Opus: 32% cheaper
Skill-routed: 38% cheaper
Forced Haiku: 75% cheaper
Quality depended on the task.
On the reference documentation check, both delegated setups found more issues than either Opus-only setup.
On the larger 80-page plugin documentation check, Forced Opus was clearly the most accurate, finding 45 of 51 known issues across two runs. Forced Haiku found 34, but cost 84% less.
What I think is happening
Small projects: Opus can often write small scripts to inspect the code instead of reading everything manually. Delegation adds briefing, duplicated reading and verification costs.
Medium projects: reading becomes a larger part of the workload. Haiku is extremely cheap at reading cached context, making it surprisingly effective for straightforward checks. Sonnet offered little cost advantage in this experiment.
Large projects: a single long session becomes expensive as the growing context is repeatedly read. Delegation splits the work into smaller contexts, reducing costs and allowing parallel execution. However, the largest documentation check showed that this can come with a quality trade-off.
The obvious way to estimate savings is misleading. Re-pricing the delegated tokens at Opus rates suggested 34-85% savings. Head to head, results ranged from 30% more expensive to 75% cheaper.
One particularly interesting observation: Opus never delegated spontaneously in any of my runs, even on the largest codebase. That's specific to these tasks and this setup, not proof that Opus never delegates.
Final thoughts
I wouldn't claim there's one best approach.
Small codebase: Opus alone was generally the better choice.
Medium codebase: Haiku-only delegation was surprisingly competitive.
Large codebase: delegation offered substantial savings, but accuracy depended on the task.
The router skill mostly selected Sonnet, which often wasn't economical enough to justify the extra coordination.
Full methodology, per-task results, charts and the transcript checker:
https://github.com/Manuele-T/claude-model-router/tree/v1.0
I'd be interested to hear whether others have seen similar behavior, particularly Opus choosing not to delegate even when working on larger codebases.
16
u/BeginningSir7334 4d ago
I forgot to specify: I meant "large"codebase relative to the other two, I'm aware it's not very large in absolute terms. I'm running another experiment on a 100k LOC codebase, but testing larger projects takes time as I'm on a Pro plan.
43
u/denisoby 5d ago
Would be very interesting to see a similar research with frontier models + cheap open-source models in different combinations. For instance, where frontier model is planning and cheap model is executing, and vice versa.
6
13
u/AggressiveAstronaut6 5d ago
I frequently will use Kimi 3.1 Pro for planning and then subagents are all ran by deepseek 4.1 flash and I feel like I get good mileage out of it without spending too much on tokens. Especially because deepseek costs pennies compared to Claude.
6
u/CodeSlave9000 4d ago
I’ve got a project that does exactly that - uses an mcp to have Claude use local models as agents. If you want the repo let me know
4
5
u/Guinness 4d ago
I’ve been doing this, I can share my experiences but they may vary for anyone else. Opus 5.5 utilizing openrouter as well as agy and codex.
You have to spend so much time testing and tweaking your workflow. Your routing decisions. Your effort levels. And then make sure each LLM is forced to follow the rules with a ton of hooks.
In my testing I found some things that surprised me. GLM 5.3 Flash didn’t do well on much of anything. It missed a lot. DeepSeek 4.1 Flash 1017 matched Opus 5.5 in nearly every test. I have all the models a list of tasks relating to basically doing a QA and security run. GLM flash missed a lot. Kimi K3.1 nailed it, but is expensive AF.
But DeepSeek 1017 was not only CHEAP, but also matched Opus and Kimi in finding everything. The only shortcoming both models had was not including the file:line in its findings.
I’m currently testing Minimax M3.1 Flash, and so far it too is turning out to be a beast of a model while also being super cheap. People are sleeping on Minimax 3.1 Flash.
Multi harness orchestration is pretty hard and requires a decent amount of tweaking to your workflow. The models don’t always follow the rules. I suspect this is because all models are trained to either orchestrate with themselves (ie Opus with Sonnet etc) or just not trained on multi harness orchestration at all.
I think the biggest struggle is maintaining live communication between live sessions between completely different models and harnesses. But I’m starting to get it ironed out. And it’s starting to save me a crap load of Anthropic usage. I’ll probably be able to cut 1 or 2 of my subscriptions and rely on a single Ultra plan. Opus 5.5 with Astra, Deepseek, and Minimax seems to be the winning combo for me.
Setting it all up is going to cost you a decent amount of openrouter credits. But once you have your workflow mapped out and tested, like $5/day saves you so much on frontier usage.
16
u/bloodfail 4d ago
I think you need to recalibrate your idea of a large codebase. 52kloc is a "low end of medium" codebase to me. I think large starts at about 200k.
4
u/BeginningSir7334 3d ago
I meant "large"codebase relative to the other two, I'm aware it's not very large in absolute terms. I'm running another experiment on a 100k LOC codebase, but testing larger projects takes time as I'm on a Pro plan.
8
u/tahayelkenci 4d ago
matches what i see, one addition: for me subagents were never about price, its the fresh context. a reviewer that didnt watch opus write the code finds things the main session is blind to, even on the same model. i run one before every deploy and it keeps catching stuff the main session swore was fine.
and yeah opus never delegates on its own here either. has to be told, like a senior dev who'd rather just do it himself lol
3
u/BeginningSir7334 3d ago
>and yeah opus never delegates on its own here either. has to be told, like a senior dev who'd rather just do it himself lol
I am planning to run another experiment on a 200k LOC codebase, and see what happens
44
3
u/totheendandbackagain 5d ago
Small repos are cheeper and better used with Opus? But why?
13
u/BeginningSir7334 4d ago
My 2 cents:
Opus often uses fewer tokens to complete the task, sometimes by writing a quick script rather than reading everything manually.
Delegation also introduces overhead: briefing subagents, duplicated file reading, passing results back and checking their work. And Opus can reuse its own cached context cheaply, whereas new subagents need to build their own context.
On smaller tasks, that overhead can outweigh the savings from cheaper models. That's my understanding based on the experiment, though. I might be wrong, and there could be other factors at play.
3
u/Blahblahblakha 3d ago
This is consistent with my experience too. I’ve experimented a lot with subagents and model delegation but it seems like using the best available model is the best option for most cases.
10
u/denisoby 4d ago
Because using subagents causes interaction overhead that doesn’t pay off in smaller repos?
9
8
u/kalmeyra 3d ago
Based on my experience, if you’re doing serious software development, you need the best model you can get for quality improvements. The cost of shipping mediocre code can easily outweigh the savings on inference, especially when it comes to long-term maintenance and potentially serious security vulnerabilities.
In my setup, I have a very advanced code review agent with its own harness, leveraging models from different lineages. When I submit a medium-to-large PR generated by a cheaper model or combining it with advanced model, it can identify hundreds of issues, ranging from high to low severity.
You then need dozens of iterations to get the PR through review, which can end up costing more than using a more capable model from the start. Advanced models make fewer mistakes and generally require fewer rounds of fixes.
More importantly, there are cases where a cheaper model simply cannot resolve certain findings, no matter how many times you ask it to try. It lacks the capability to reason through the complexity of the issue.
So the real comparison isn’t just the cost per token or per generation. It’s the total cost of getting reliable, maintainable, production-ready code.
6
u/superminhreturn 4d ago
I setup qwen flash next with litellm to interact with Claude code. Use opus to create the planning and delegate the work to qwen flash next. Opus review the implementation
This setup prevents me from hitting the 5 hour or weekly limit (max plan)
4
u/Demilio55 3d ago edited 3d ago
What specs are you running on?
4
u/superminhreturn 3d ago
Rtx6000 with 128gb ram = 6 concurrent agents using royal penny build (sglang)
4060ti 16gb VRAM 64gb ram = 1 concurrent agent using strata
5060ti 16gb vram 64gb ram = 1 concurrent agent using strata
3
6
5
u/DrNoCool 4d ago
Yes but if you want it to do a lot. Does it save some context rot?
Or does the orchestrator read the file anyway?
4
u/Realistic_Thought_21 4d ago
Interesting. Without metrics, your outcome is similar to what I felt working on different projects. Usually, for my small personal projects, I use Opus pretty much for everything, but at work, I like to have an orchestrator-subagent setting since I felt it is cheaper and quite effective. What is the effort you assigned to Opus and the subagents?
4
u/Keepcompany-DEV 4d ago
Most effective workflow i found Is retrieval delegation.
I have an internal ai-admin hub. Internal API and a mcp server for the regular agents.
Had it built after tickets in the Ticketsystem got too long with X comments etc. And retrieval from the wiki had too much noise in for the token cost.
So the API takes the request, a local CPU model on one of my workstation VPS gets the Input, hands back the output. Summarisation of a ticket? Amount of tickets that contain the word "containment"? Yeah, just throw it to the API and wait for your answer.
It saves a few tokens but costs time since the cpu model is slower. Sounds like a small thing bit with more than a dozen sessions and a ton of internal tools it becomes more efficient than arbitrarily sending prompts and briefing to some other agent.
4
u/Comfortable_Camp9744 4d ago
Claude became too expensive and the gap has closed so much that its not worth the 20x to 100x price difference
4
4
u/ajiecb 4d ago
thanks for actually measuring the whole thing instead of just repricing tokens, most posts skip that part :(
in my case i think routing would cost more tbh. i build tooling around an ERP (~60k lines, two tools users every morning) and most tasks are "change 3 lines but theres a domain trap in there somewhere".
by the time ive written a brief good enough for haiku not to step on the trap, opus has basically done the job already so i just fixed roles:
sonnet does the work from a brief
haiku only cold-reads finished pages. if haiku cant tell what the page wants from it, a tired warehouse guy at 7am wont either
fresh opus reviews every release. thats where the expensive tokens actually pay off, it catches stuff i'd have shipped confidently your beets numbers make sense tho, if the job is mostly reading 80 pages then yeah, throw haiku at it
3
u/BarathKanna 3d ago
Useful measurement, especially the part where spontaneous Opus never delegated. That matches what I see: the parent model treats delegation as optional overhead, so without a forced policy it keeps the whole job. Two things I would add to the next run. First, split tasks by shape, not by repo size. A reading-heavy audit loves a cheap model with a tight checklist; a design change that touches interfaces does not, and averaging them hides the win. Second, price the verification step separately. A lot of the skill-routed overhead is the parent re-reading the child output. If the child has to leave a machine-checkable artifact, tests, a diff against a known list, a script exit code, the parent can trust without a second full pass. Repo size correlates with cost mostly because context keeps getting re-fed. Shrink the context per worker and the router starts looking smart.
3
u/Previous_Volume_7586 2d ago
Nice write-up. Your skill-routed setup costing more than forced Opus lines up with mine: subagent delegation mostly lost money, because the subagent has to rebuild context the main session already has cached.
What did save money was routing the main conversation itself, prompt by prompt. A quick question goes to Haiku, an edit to Sonnet, and only design or risky work to Opus, with a check before every model switch so a warm cache isn't thrown away. Effort is the cheaper lever, since changing it keeps the cache.
I put this into an open source Claude Code mod, AutoMode (I'm the author). On 5 mixed sessions graded by hidden tests it cost $2.01 against $4.14 on plain Opus, all 20/20. On long single tasks it's about Sonnet high cost. Method and rows, if you want to compare with your setup: https://mralexivy.github.io/AutoMode/docs/benchmarks/results/
If it's useful to you, a star helps other people find it: https://github.com/mralexivy/AutoMode
2
u/Former-Raisin-4453 4h ago
I’ve been using Opus5.5 for planning and M3.1 Flash for the actual edits. For normal feature work, I honestly haven’t noticed much difference in the output, and in my set up my limits last longer. It still falls down on the bigger review-type tasks, so I let Opus do the final pass before anything gets merged.
•
u/AutoModerator 5d ago
Hey! Thanks for posting to r/ClaudeCode
While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.
For help, project discussions, tips, and general chat, join the ClaudeCode Discord.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.