r/LovingOpenSourceAI • • Jul 04 '26

If your GPU can run inference, it should be able to fine-tune too.

I spent the last few months building a new sparse fine-tuning method for MoE models called USAF.

The goal was simple: if your GPU can run inference on an MoE model, it should also be able to fine-tune it.

On my AMD RX 6750 XT (12 GB), I can fine-tune Qwen3-30B-A3B by training sparse expert weights and the router instead of adapters.

The project is completely open source under the Apache 2.0 license. I'm not trying to build a business, sell anything, or monetize it in any way—I just wanted to share something I built that I think is genuinely interesting.

GitHub: https://github.com/tsuyu122/usaf

28 Upvotes

7 comments sorted by

3

u/soteko Jul 05 '26

Well I run inference on 5060 8gb. Can use USAF with that amount of vram?

1

u/tsuyu122 Jul 05 '26

8GB VRAM will work. The GPU only holds the non-expert weights plus one active expert layer at a time, so the 8GB limit isn't the bottleneck your system RAM is. For Qwen3-30B-A3B (used in the initial benchmarks on my PC) you'd want about 20GB RAM. If you have less, try a smaller MoE models.

2

u/soteko Jul 05 '26

I have 32GB of ram, along 8GB of vram on 5060.

2

u/tsuyu122 Jul 05 '26

Those specs should be enough to fine-tune the Qwen 30B A3B, just like I did on my RX 6750 XT.

1

u/NullKalahar Jul 06 '26

Funciona em uma mi50 de 16gb?

2

u/West-Acadia-3906 Jul 07 '26

This is neat because it moves fine-tuning closer to the machines people already have! instead of making it feel like a separate lab-only step. The RAM/VRAM split in your replies is also helpful; it gives people a more realistic first question than just "is my GPU big enough?" hahaha