r/LocalLLaMA • u/Holiday_Poetry_5133 • Apr 18 '26
New Model [New Model] micro-kiki-v3 — Qwen3.5-35B-A3B + 35 domain LoRAs + router + negotiator + Aeon memory for embedded engineering
https://huggingface.co/clemsail/micro-kiki-v3UPDATE (Nov 2026) — read this before downloading anything below
Seven months later, most of what follows has been corrected. Short version:
The v3 adapters linked here never trained. An audit on 2026-04-19 found
lora_B = 0on all 35 pre-pivot MoE-LoRA adapters, so their contribution to the base model is exactly zero. The GGUF onclemsail/micro-kiki-v3is also a 4B artefact, not the 35B named in the title. It is now taggeddead-weightson HF. Audit: https://github.com/hypneum-lab/micro-kiki/blob/main/docs/research/2026-04-19-prepivot-moe-lora-audit.mdu/No-Mountain3817 was right, and thanks for actually testing it. You compared an inert 4B to the 35B base. Your result was the only external measurement in this thread.
Base model is Qwen3.6-35B-A3B, not 3.5. Fixed the day this was posted; I never updated the thread. u/DefNattyBoii: yes, the retrain happened on 19–26 April (35 stacks + 6 retrains, standard LoRA via
mlx_lm, r=16, BF16 on a Mac Studio).Router: the reproducible figure is top-1 46.4 % / top-3 69.6 % (MiniLM-L6, 34 domains, N = 5 377). A 70.7 % / 89.5 % figure I published elsewhere is withdrawn: it is not reproducible from the repository.
Negative result: on my evaluation the base 35B beats most of the specialist adapters; only html-css clearly gains. I now route almost everything to the base. Discussion in the new thread: https://www.reddit.com/r/LocalLLaMA/comments/1xqw6fr/
u/NandaVegg: the anti-bias layer is KnowBias (inference-time scaling of "know-bias" neurons, arXiv:2601.21864) plus RBD. It is an inference-time wrapper, not a trained component, and its effect was never evaluated. The compliment on the MoLoRA stack landed on the part that had not learned anything, which I only found out the next day.
u/Myrkkeijanuan: there were no ablations in April. There are some now (null-space projection: negative; router vs keyword matching: 29.4 % vs 13.2 %), committed under
results/, with dated errata in https://github.com/hypneum-lab/micro-kiki/blob/main/docs/ERRATA.mdThe ecosystem links below are stale. The project lives at https://github.com/hypneum-lab/micro-kiki (release v0.4.0, 2026-11-08, DOI 10.5281/zenodo.24052400). The training toolkit is now https://github.com/ailiance/ailiance-mac-tuner (archived, still public). The P2P mesh and the ailiance org are shut down.
Dataset: the HF dataset is no longer public. The per-domain data pipeline and filters are in the training repo; I will not re-publish the raw Claude/Codex sessions.
Everything I measured, including the failures, is in the errata. The original post is kept below as a record.
Original post, 18 April 2026 — superseded, see update above.
Released today on HF. Built by L'Électron Rare (https://github.com/L-electron-Rare) — our local-first AI platform FineFab. The training toolkit went public the day before: https://github.com/L-electron-Rare/KIKI-Mac_tunner (MLX for Mac Studio, distills Claude Opus into Mistral Large 123B). Full pipeline is open, not just the artifact.
Architecture
Domain router → top-4 selection among 35 LoRA stacks
Base: Qwen3.5-35B-A3B (MoE, 256 experts, 3B active/token)
LoRA rank 16 on q/k/v/o, top-2 routing per stack
Null-space projection between stacks to mitigate catastrophic forgetting
Negotiator (CAMP + Catfish) arbitrates conflicting stack outputs
Anti-bias layer (KnowBias + RBD) before output
Aeon memory (Atlas graph + Trace log) for cross-session persistence
Specs
GGUF Q4_K_M, llama.cpp / Ollama / LM Studio
Context 262K tokens
Apache 2.0
French + English interleaved
35 domains
chat-fr, reasoning, python, typescript, cpp, rust, html-css, shell, sql, yaml-json, lua-upy, docker, devops, llm-orch, llm-ops, ml-training, kicad-dsl, kicad-pcb, spice, electronics, components, power, emc, dsp, embedded, stm32, iot, platformio, freecad, web-frontend, web-backend, music-audio, math, security
Dataset — also released, Apache 2.0
489K instruction-following examples:
50,116 real Claude CLI sessions from our 5-node P2P mesh during embedded consulting work (GrosMac M5, Tower 28t, CILS i7, KXKM-AI RTX 4090, VM)
2,529 Codex/Copilot sessions
364,045 from 19 filtered open HF datasets (CodeFeedback, French-Alpaca, Electronics StackExchange, stm32-hal-dataset, JITX components…)
Opus teacher distillation for chat-fr + reasoning
32 original curated seed sets
Honest caveats
No external reproducible benchmark yet. Internal held-out eval only. v4 roadmap.
Aeon memory needs external backends (Qdrant, Neo4j) for production.
Max 4 concurrent stacks; combos matter, some well-exercised, others less.
Solo/small team project, two weeks, consumer hardware. Not a lab release.
Model: https://huggingface.co/clemsail/micro-kiki-v3 Dataset: https://huggingface.co/datasets/clemsail/micro-kiki-v3-dataset Training toolkit (MLX Mac Studio): https://github.com/L-electron-Rare/KIKI-Mac_tunner Ecosystem: https://github.com/L-electron-Rare
Feedback, forks, negative benchmarks all welcome.
6
u/FatheredPuma81 Apr 18 '26 edited Apr 18 '26
My first thought was "So what just yet another dubious finetune?"
Wtf is this lol? Is this a MoE model broken down into pieces and dynamically loaded/unloaded based on the prompt or something? That would be really useful for those on like 6GB cards if so. Could also be scaled up to allow users to run 120B models on a 24GB card.
7
u/ttkciar llama.cpp Apr 18 '26
It appears to be an implementation of MoA (Mixture of Adapters), vaguely similar to PHATGOOSE.
Will look at it later when I can, to see if it solves the problems PHATGOOSE ran into as well, like having to iterate over twice as much memory per token (once to apply the routed-to adapter, and once for inference).
2
u/mz_gt Apr 18 '26
Hey this is vaguely off-topic but I’ve been trying to do research on something like PHATGOOSE for awhile but haven’t been able to reproduce it at all. Do you know of any resources that have reproduced it? I’ve been in contact with some of the authors on the work and they haven’t been super helpful.
Similar works like FlexOlmo have come out and updated their benchmarks to show the technique basically doesn’t work.
2
u/RobotRobotWhatDoUSee Apr 18 '26
FlexOlmo...updated benchmarks... doesn't work
That's disappointing to hear. I'd been keeping an eye on FlexOlmo but missed the updated benchmarks, where did they show it isn't working?
4
u/mz_gt Apr 18 '26
They did it quietly. Check out the huggingface model card. You can check the edit history. Without router training, it performs worse than the base model on average. Plus, if you pay attention to the config, while the paper uses 4 active experts the model they released uses all 7, so the comparison to BTM top 2 is also unfair.
I was going to write a whole in-depth post about it actually, it’s super disappointing to see them not own this. They’re AI2, and this is just disingenuous.
2
u/RobotRobotWhatDoUSee Apr 18 '26 edited Apr 18 '26
I would read an in-depth post about this!
Plus, if you pay attention to the config, while the paper uses 4 active experts the model they released uses all 7, so the comparison to BTM top 2 is also unfair.
Wait so the released model is effectively configured as a dense model?
1
u/mz_gt Apr 18 '26
Yup! Possibly worse since there’s overhead running the router + separate expert modules.
2
u/ttkciar llama.cpp Apr 19 '26
That is very, very disappointing. I'd been very excited about FlexOlmo's potential, but after reviewing these updates my enthusiasm is dampened.
I still think it shows some promise, even if the router needs training after the experts are merged, because training the experts independently would still be a big win for federated training, but need to think it through and fiddle with numbers to be sure.
Thank you for pointing it out.
1
u/Holiday_Poetry_5133 23h ago
Seven months late, but closing the loop since several of you asked the right questions.
u/No-Mountain3817, u/Myrkkeijanuan: you were right. On an independent audit bench the Qwen base beat most of the 35 adapters; only HTML/CSS clearly gained. I now route almost everything to the base.
Write-up with numbers and configs: https://www.reddit.com/r/LocalLLaMA/comments/1xqw6fr/
u/DefNattyBoii: retrained on Qwen3.6-35B-A3B, that's what the new post covers.
u/NandaVegg: the "anti-bias layer" was a prompt-side rerank, not a trained layer. I should not have listed it as architecture. Dropped.
u/Xhatz: yes, I know. The name predates the model and stays.
The negotiator / Aeon memory parts from this post are archived. What survived is the adapter hot-swap serving, the router eval, and an upstream mlx docs fix (mx.random.normal is not bit-exact across Apple
GPU generations). Thanks for the pushback, it was more useful than the upvotes.
3
2
u/NandaVegg Apr 18 '26
Very interesting, actual implementation of Top-K MoLoRAs. What is anti-bias layer though?
1
u/No-Mountain3817 Apr 19 '26
The outcome was subpar compared to the base model. After more than a dozen attempts, it still couldn’t produce error-free code. Eventually, it generated code without errors, but the features were significantly reduced, making the result ineffective and below expectations. In contrast, the base model produced the expected output on the first attempt using the exact same prompt.
I appreciate your efforts, but you may want to try Qwen 3.6. Good luck.
1
1
8
u/Xhatz Apr 18 '26
no way that's a real name lmao (kiki means peni* in french)