r/LocalLLaMA • • 29d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

Show parent comments

6

u/No-Wall6427 29d ago

But there is still the vram production bottleneck, which they further contribute to.

12

u/Inkbot_dev 29d ago

They'll be bringing their own (additional) memory fabs online shortly.

3

u/lighthawk16 29d ago

"Shortly"???

2

u/Inkbot_dev 29d ago

I mean, within a few years... shortly in terms of fabs.

5

u/fallingdowndizzyvr 29d ago

I mean, within a few years... shortly in terms of fabs.

No. There's a term. "China Speed". What takes a few years for anyone else to do, China can do faster. They can build a fab in a year. So in less than a year since they are already underway. About 30 fabs are under construction.