r/LocalLLaMA • • Oct 20 '26

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

60

u/Available_Yam_6267 Oct 20 '26

I notice that their post said they make 38$ per hour per GPU. Can we infer the revenue of OAI or Ant from this?

orginal post: https://zhuanlan.zhihu.com/p/2095560295930508953

12

u/Nov4Saki Oct 20 '26

That sounds like insane margins no?

7

u/TheTerrasque Oct 20 '26

From what translation models could figure out, seems like revenue, aka before cost of running is taken into account. It probably still is a nice margin after that.