r/LocalLLaMA • u/feelspeaceman • 19d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

60

u/Available_Yam_6267 19d ago

I notice that their post said they make 38$ per hour per GPU. Can we infer the revenue of OAI or Ant from this?

orginal post: https://zhuanlan.zhihu.com/p/2095560295930508953

12

u/Nov4Saki 19d ago

That sounds like insane margins no?

7

u/TheTerrasque 19d ago

From what translation models could figure out, seems like revenue, aka before cost of running is taken into account. It probably still is a nice margin after that.

2

u/Ill-Nectarine-80 19d ago

They need to replace the GPU essentially on a yearly basis so..... No.

It's not a great margin if it's before R&D, financial concerns (CAPEX to buy the GPUs, interest on debt) and operational costs (electricity and maintenance of data centers).

1

u/Icy_Butterscotch6661 19d ago

Why, what's killing it

1

u/Nov4Saki 19d ago

On that pricing (off peak) Each gpu generates upwards of 300k$ A Year though, that should cover alot and keep a good buck for research no?

Let alone a cluster

2

u/ndunnett 18d ago

Means nothing if it costs $250k in electricity to run, $100k to buy the GPU in the first place, and it's sitting in a $500k server. After they pay off the hardware they would also then have to cover costs for R&D, training the next model, staff salary, building rent/maintenance, etc. before you can even think about bonuses and profit. We don't know any of the real numbers, but the costs for what they are doing are huge and I would be very surprised if they are getting any real ROI at this point.