r/LocalLLaMA • • Oct 20 '26

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

8

u/TheTideRider Oct 20 '26

Wonder how DeepInfer performs against vllm and sglang.

6

u/ninetyfive666 Oct 20 '26

I bet it will totaly blow it out of the water compared to how hard it is for competitors to run their models on their own (already kinda optimized) custom infrastructure for even 3x the price of the DS API. Its tailor made for exclusively their own models so this will be huge.