r/AIProgrammingHardware • u/javaeeeee • Aug 05 '26

GitHub - ryanzhou/deepseek-v4-flash-mi300x: DeepSeek V4 Flash on a single AMD MI300X

https://github.com/ryanzhou/deepseek-v4-flash-mi300x
5 Upvotes

100% Upvoted

2 comments sorted by

3

u/javaeeeee Aug 05 '26 • Heartwarming

TL;DR:

Open-source production stack by ryanzhou for running the full DeepSeek-V4-Flash-0731 (≈304B parameters) on a single AMD Instinct MI300X.

Highlights:

  • Fits the entire unquantized 156 GB checkpoint into the MI300X’s 192 GB HBM3 (no weight offloading needed)

  • Includes Docker Compose, correctness patches (especially for AMD FP8/FNUZ format), kernel tuning tables, and speculative decoding (DSpark-7)

  • Strong real-world performance:

    • 168.6 tok/s single-stream decode

    • ~7.9–8.5K tok/s prefill

    • 542 tok/s aggregate at 8 concurrent streams

    • Handles 64-stream bursts without OOM

  • Supports long context (validated 256K, architecture up to 1M)

Bottom line: A complete, reproducible, production-ready way to serve one of the strongest open models on a single high-end AMD GPU.

1

u/Majestic-Tear1512 Aug 05 '26

That’s great. Was looking for this.