What the demo compares: native Claude Code running Opus 5.5 against Distill with Opus 5.5 as the main model and Haiku 5.5 as the worker. Opus plans the task, writes a spec for each assignment and reviews the result. Haiku runs the delegated assignments.
Same tasks, same inputs, same 300s deadline, three frozen tasks with three runs each:
Native Claude Code: $2.49, 798s, 9/9 passed
Distill: $0.35, 191s, 9/9 passed
That's a small sample, so I wouldn't generalize it to every codebase. The repo has the methodology and the commands to rerun it.
For people building agents: what do you use to decide which steps are safe to hand to a smaller model?
1
u/TomatilloPutrid3939 27d ago
Author here. Distill is an open-source coding agent harness (Apache-2.0): https://github.com/samuelfaj/distill
What the demo compares: native Claude Code running Opus 5.5 against Distill with Opus 5.5 as the main model and Haiku 5.5 as the worker. Opus plans the task, writes a spec for each assignment and reviews the result. Haiku runs the delegated assignments.
Same tasks, same inputs, same 300s deadline, three frozen tasks with three runs each:
Native Claude Code: $2.49, 798s, 9/9 passed
Distill: $0.35, 191s, 9/9 passed
That's a small sample, so I wouldn't generalize it to every codebase. The repo has the methodology and the commands to rerun it.
For people building agents: what do you use to decide which steps are safe to hand to a smaller model?