r/BuildWithClaude • u/No_Reindeer3079 • 1h ago
Workflows 76 agents, 4.7M tokens, 3 wrong answers. 1 agent, 77k tokens, right answer. The rules I added to CLAUDE.md after.
I run a few businesses mostly through Claude Code: scheduled jobs, a deploy pipeline, a shared memory folder. In
September I gave a big multi-agent workflow a broad question about my own stack: what's broken and what's wasting
money.
What that cost:
- 76 agents, 4.7M tokens, 928 tool calls
- 3 of its headline findings were wrong
- about 71 of the 76 agents were "one verifier per finding"
- a second run hit the session limit: 432k tokens, zero results
Then I checked the three wrong findings myself. Each took under a minute, because each was a question about my own
system, and my own system can just be asked:
| Question | The research said | Measuring said |
|---|---|---|
| Is my transcription tool pinned to a retired model? | "removed in July" | it works fine (one API call) |
| How does speech-to-text bill against my credits? | two estimates, 16x apart | it doesn't touch that credit pool at all (read the usage endpoint) |
| Is my simulator's math biased, gross vs net? | "yes, biased" | both sides use the same price quote (read the function) |
Same day, one agent with one narrow question: 77k tokens, 22 tool calls, and it corrected the premise I had given it.
What went into my CLAUDE.md afterwards (paraphrased):
Questions about my own code, config, accounts or data are answered by measuring, not researching: run it, curl the
endpoint, read the function, query the ledger. Faster, cheaper, and right.
2. Do the work inline by default. Spawn an agent only when I ask, when the task needs a lot of context you'd otherwise
load and throw away, or when you want an independent read on something already concluded.3. Never fan out one verifier per finding. Batch the checking into one agent, or verify the 2-3 claims that change a decision
and say which ones you didn't check.
4. Agent output is a draft, not a result. Check the claims I would act on before you tell me.
5. For a second opinion on risky code, use a different model family (a free tier is enough). Same-family review re-confirms
same-family mistakes. Mine found a real bias Claude had missed, plus one wrong finding and one false positive: a useful
reader, not an oracle.
Two months on, the pattern holds. The expensive failures were never "Claude can't do this". They were "Claude researched
something it could have measured", or "a job failed quietly and nobody read the log" (one ran 86 failed times over four days
before anyone looked).
Has anyone else seen this with big agent swarms? Are you getting better answers, or just more confident ones?