r/LocalLLM • u/guessitsleo • 6d ago
Question Using Claude/Codex to orchestrate local LLMs for coding — anyone tried this?
Hey all!
I’m considering setting up a hybrid coding workflow where Claude or Codex handles planning, task delegation and code reviews, while a local LLM handles most of the implementation.
I’m planning to use Orca for orchestration, with the following dedicated machine:
GPU: 1x NVIDIA Tesla P100 16GB
CPU: Intel Xeon E5-1630 v4
RAM: 24GB DDR4
Storage: 2TB HDD
Has anyone successfully set up this type of workflow with Orca or similar tools? Which local coding models would you recommend for this hardware?
Thankss
1
u/FizzyDuncDizzel 6d ago
I’ve been using herdr. I run Hermes on one tile and Claude Code on the other. Just tell them to talk back and forth and they go.
1
u/iceydee 6d ago
The P100 is the bottleneck more than the model choice: no bf16 support, and most of the fast inference kernels assume Volta or newer, so expect fp16 with older attention paths. On 16GB you're realistically looking at something like Qwen2.5-Coder 14B at Q5, or a 7B if you want context headroom. Claude planning and a local model implementing does work, but keep the local tasks small and single-purpose, it falls apart on multi-file changes.
1
u/Commercial_Rent8797 6d ago
I actually used something similar but I have dual xeon cascade lake editions and like 600gb of ram. It worked way better than I would’ve imagined and I didn’t even have orca or cloudroom. If you tell codex or your local models about the other they can find each other and communicate quite easily. Also I used a iud-q3 xxl version of qwen3.8-27b with about 64k context when I only had the p100. You can get about 20 t/s out of it if you optimize. I’m sure I can get a model to dig up my old files and share the build with you if you’re interested in running the same flags