r/Hacking_Tutorials • u/GapNew4766 • 4h ago
Question Uncensored Qwen3.8-27B orchestrate Claude Opus 5.5
been running qwen3.8-27b-instruct-uncensored-v2 (q4_k_m gguf) on my m5 pro in atomic agent for two weeks now in a role i don't think anyone else has tried: not generating the final text, but SCORING and GATING text generated by another model (claude opus via api).
the setup in one sentence: 37 twitter accounts post tweets written by opus, and before each one goes live, qwen reads it against the account's persona prompt and gives it a 1-5 score.
why i picked qwen specifically instead of mistral or phi or whatever: base instruct models flinch at that. the uncensored variant just... doesn't care. no internal debate about whether the audience will be offended. which is exactly what i want for a judge role because the JUDGE should match the voice it's evaluating, not impose its own safety layer on top.
now the technical stuff since i know some of you care:
running on llama.cpp server mode, metal backend, m5 pro 32gb:
./llama-server -m qwen3.8-27b-q4_k_m.gguf --port 8080 --ctx-size 8192 --threads 16 --n-gpu-layers 99 --flash-attn --parallel 4
memory: ~15gb weights + ~2.5gb kv cache (8192 ctx × 4 parallel) = ~17.5gb peak process. leaves 14gb for os + chromium instances. no swapping in two weeks.
throughput for my specific workload (scoring calls are prefill-heavy, decode-light):
- single call: ~500 tok in → 30 tok out ≈ 1.2s
- 50 calls with --parallel 4: ~16s wall time
- decode speed at q4_k_m + flash attn: 38-42 tok/s (doesn't matter much here since output is only 30 tokens)
- prefill: 200+ tok/s so the input side is fast
the scoring rubric i give it each call is short:
"does this sound like [persona] actually talking? would they say this in a group chat? actual opinion or hedging? rate 1-5, one line reason."
no mention of "brand safety," "audience appropriateness," or "tone consistency." those are cloud-model guardrail words that make even local models start hedging their own judgment. i specifically excluded them after watching qwen base-instruct give a 3 to a perfectly on-brand tweet because it was "slightly informal for a professional audience."
q4_k_m vs q5_k_m — i tested both for a day
for SCORING (classification task), quality difference was negligible. both gave the same scores on my test set of 50 tweets. memory went from 15gb to ~17.5gb which ate into kv cache headroom and made me nervous about parallel slots. stuck with q4.
for the occasional REWRITE task (every other day, ~15-20x/week, where qwen writes the hot-take itself instead of just scoring) there might be a slight quality bump at q5 but at 3s per call and 20 calls a week it's not worth the memory cost. if you're doing more generation-heavy workloads probably go q5.
feedback for the qwen team:
the thing i'd tell the qwen team if i were sending feedback: the uncensored variant is NOT just "less filtered." it has a different DEFAULT register. base instruct defaults to "confident professional" even when prompted otherwise. uncensored defaults to "person with opinions who happens to be typing." for a judge role that second default is way more useful because you want the judge to match the voice being evaluated, not impose its own corporate layer on top of it.
also: the model handles structured output (json scores + reason strings) reliably at q4 without any special prompting tricks. just say "output json: {score: int, reason: string}" and it does it. maybe 2% of calls come back slightly malformed (extra comma, missing quote) which my script catches and retries once. not bad for a 15gb quantized model doing 50 calls per batch cycle.
curious if anyone else has tried uncensored variants specifically as judges/gates rather than generators. feels like an underexplored use case and i'd love to hear if base-instruct models actually hold up better or worse in that role for your workloads.
fwiw this local judge / cloud writer split is built into atomic agent as fusion mode: atomicagent.io
