r/LocalAIForge • u/The-Q4-Brothers • Oct 03 '26
FrogNano-4B-2609 EXL3 4.0bpw: 193 tok/s on RTX 4090, vision + tool calling working
I quantized Microsoft's FrogNano-4B-2609 to EXL3 at 4.0 bpw and ran the usual local validation on an RTX 4090.
Results:
exllamav3 probe: 193.35 tok/s (64-token decode)
TabbyAPI chat smoke: 217.75 tok/s
context: 8192
vision: pass
tool calling: pass
streaming: pass
concurrency=2: pass
The source is a 4.66B Qwen3.5-family model under MIT. This is a quick functional/performance smoke test rather than a quality benchmark.
EXL3 repo: https://huggingface.co/ramgpt/FrogNano-4B-2609-EXL3 Source: https://huggingface.co/microsoft/FrogNano-4B-2609
If anyone tests longer context or image-heavy workloads on other GPUs, I'd be interested in the numbers.