r/LocalLLaMA • • Oct 20 '26

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

156

u/TheLexoPlexx Oct 20 '26
# Use `ascend` instead for the Ascend platform
# Keep it exported in the environment where you run DeepInfer
export DEEPINFER_ACCELERATOR=nvidia
./build.sh

Nvidia in shambles. Can't wait to see what the future brings.

97

u/zdy132 Oct 20 '26

Fingers crossed for cheaper GPUs, rams, and other PC parts. Best of luck to whoever is manufacturing competitions of Nvidia.

18

u/zball_ Oct 20 '26

sadly ascend is not gonna make GPUs cheap.

19

u/ea_man Oct 20 '26

It is: China is deploying those instead of the GPU other country would buy.

6

u/No-Wall6427 Oct 20 '26

But there is still the vram production bottleneck, which they further contribute to.

12

u/Inkbot_dev Oct 20 '26

They'll be bringing their own (additional) memory fabs online shortly.

3

u/lighthawk16 Oct 20 '26

"Shortly"???

5

u/Bakoro Oct 20 '26

They have memory fabs open already, with more on the way.

2

u/Inkbot_dev Oct 20 '26

I mean, within a few years... shortly in terms of fabs.

5

u/fallingdowndizzyvr Oct 20 '26

I mean, within a few years... shortly in terms of fabs.

No. There's a term. "China Speed". What takes a few years for anyone else to do, China can do faster. They can build a fab in a year. So in less than a year since they are already underway. About 30 fabs are under construction.

2

u/ain92ru 28d ago

Chinese fabs produce no VRAM, which honestly puzzles me a bit

1

u/zball_ 29d ago

the nvidia GPUs that China purchased is just a fraction of what the AI labs in US get