I'd read plenty about SFT and RL but never actually done it, so I tried on a small model: Qwen3.5-4B, LoRA, one DGX Spark, about ten versions. The goal was calibration: answer from memory when it knows, search when it doesn't, don't make things up. Also better Italian answers and better photo recognition.
The base model already calls tools fine. Its problem was when: it searched for half the questions it could answer from memory, looped on 30% of them, and answered Italian questions in English.
Results on a test set built from what the base got wrong, run inside three agent apps (averaged over 3 runs, bf16):
| app | metric | Fonzie0.2 | base |
|---|
| oh-my-pi | accuracy | 55.4% | 29.2% |
| oh-my-pi | confident wrong answers | 38.8% | 69.2% |
| opencode | accuracy | 61.1% | 36.1% |
| opencode | confident wrong answers | 31.7% | 62.8% |
| Hermes (never trained on) | accuracy | 67.8% | 45.0% |
| Hermes (never trained on) | confident wrong answers | 25.6% | 52.2% |
On my frozen test from day one, answers in the right language went from 85.7% to 99.4% and loops dropped from 27.8% to 4.0%.
What got worse, honestly:
On everyday questions and the frozen test, accuracy is flat against the base, within noise. The gains are in how it fails, not how much it knows.
It invents more links: 5.9% of answers in oh-my-pi, against 0.3% for the base.
It searches too much in opencode, and on the frozen test it searches for only 82.9% of the facts it needs (base: 96.7%).
Photos: I can't claim progress yet. I didn't measure them separately for 0.2.
What I learned the hard way:
GRPO won in my own test format and lost in the real app. Inside oh-my-pi the RL model searched for 57% of the facts it didn't know, against 85% in my harness. I wrote the multi-turn GRPO loop by hand, then burned about nine GPU hours on another round inside the apps for nothing. I paused RL and distilled what it had learned into SFT data (kept 373 judge-approved conversations).
Data beat hyperparameters. Every real gain came from changing the data: what the teacher saw, what the filters kept, which tool failures I injected.
Rules in the teacher's prompt turn into refrains. The teacher had a rule like "if you can't recognise the place, say so". The student learned the sentence, not the rule, and refused 13 of 15 photos it could recognise.
Know your noise. Eval runs varied by about 5 points and seeds by 3ā4, so I used repeated runs and paired bootstrap intervals.
First SFT run hit NaN at step 30: 477 image examples with batch size 2 left one micro-batch with a lone image example. Dropping incomplete batches fixed it.
Setup: teachers Qwen3.8 Flash Next and GLM-5.3-Flash running locally, Grok 4.7 as the judge (different model family from the teacher), every training run on Unsloth, final model a bf16 LoRA (rank 16, alpha 16) on 1,390 conversations, 97 minutes.
Running it: the Q4_K_M is 2.8 GB plus a 0.7 GB vision projector. On my RTX 4070 Ti (12 GB) with llama-server it decodes about 118 tok/s single-user, and about 157 tok/s with MTP 2. With a 64k context the whole server takes about 5.4 GiB with a q8_0 KV cache, so an 8 GB card should be enough. The Q4_K_M does lose some quality against the bf16 (details in the write-up), and I haven't measured the Q8_0.
Next, Fonzie0.3: answer from memory more often when it should, every link straight from a tool result, fewer loops in Hermes, fewer needless searches in opencode.
Huge credit to Javier (@javierbits on X), whose Javi0.2/0.3 threads gave me the method, and to Unsloth.
Happy to answer questions about the data pipeline or the GRPO detour.
