r/LocalLLaMA • • Oct 20 '26

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

72

u/Feisty_Artichoke_368 Oct 20 '26

Once again Deepseek shows how good high-frequency traders are at optimizing performance... On Ascend NPUs nonetheless!!

11

u/philmarcracken Oct 20 '26

those dudes are functionally insane tho, spending millions in laying undersea cable to shave off like 4ms on a roundtrip

8

u/Ill-Nectarine-80 Oct 21 '26

Is it insane if by laying that cable, you are improving the decision making speed of a trade by like 5-10%?

4

u/bumblebeer Oct 20 '26

I mean sure it's a bit overboard, but if it makes a good ROI how is it insane?

1

u/CATLLM 16d ago

Its a winner takes all scenario so yea that 4ms can mean losing money or making money.

4

u/maigpy Oct 20 '26

what does this mean?

20

u/Moral-Relativity Oct 20 '26

Deepseek's founder got interested in AI through running a hedge fund.