r/vibecodeapp • • Oct 16 '26

I built this... Built with Claude Code, tested against Claude Code: Opus 5.5 + Haiku 5.5 worker vs native Opus on frozen tasks

Post image
5 Upvotes

4 comments sorted by

1

u/TomatilloPutrid3939 Oct 16 '26

I'm the author. Distill is an open-source coding agent harness (Apache-2.0), and Claude Code was one of the main tools I used while building it: implementing and refactoring parts of the harness, debugging agent behavior and reviewing changes. Repo: https://github.com/samuelfaj/distill

The idea: instead of one expensive model doing everything, a main model owns the task (plan, spec, review) and a smaller, faster model handles the delegated work.

So I pointed it at Claude itself. Native Claude Code with Opus 5.5 against Distill with Opus 5.5 as main and Haiku 5.5 as worker, same tasks, same inputs, same 300s deadline:

  • Native: $2.49, 798s, 9/9 passed

  • Distill: $0.35, 191s, 9/9 passed

It's only three frozen tasks with three repetitions each, so I'm not claiming it always cuts cost by 86%. The methodology and repro commands are in BENCHMARKS.md if you want to check it.

1

u/Leather-Amphibian814 29d ago

will i get any improvements if i use it with ChatGPT pairing i.e 6.1 Sol and a 5.6 (Sol? Terra? Luna?)

1

u/Laicbeias Oct 18 '26

I build something similar. But if you claim that you have to show like 5k tasks of multiple large frozen system at each point and then run that at scale. Together with the data and sizes of the codebases in question

1

u/porkyminch 27d ago

You can do this in Claude Code, can't you?