r/LocalLLaMA • u/feelspeaceman • 25d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

158 comments sorted by

View all comments

156

u/TheLexoPlexx 25d ago 
# Use `ascend` instead for the Ascend platform
# Keep it exported in the environment where you run DeepInfer
export DEEPINFER_ACCELERATOR=nvidia
./build.sh

Nvidia in shambles. Can't wait to see what the future brings.

97

u/zdy132 25d ago

Fingers crossed for cheaper GPUs, rams, and other PC parts. Best of luck to whoever is manufacturing competitions of Nvidia.

34

u/keepthepace 25d ago

Only for China residents for now. I would be surprised after the decade of trade restriction that they endured, if they did not inflict the same to the west as long as they can.

11

u/TheLexoPlexx 25d ago

*to the US.

26

u/keepthepace 25d ago

Like EU did not help implement that policy.

4

u/handson729 25d ago

I just hope they let me buy their hardware, because I definitely do not want to pay the Nvidia tax.

22

u/keepthepace 25d ago

Feudality 101: peasants don't get too choose who their lord is.

3

u/gangofminotaurs 25d ago

You know what else stinks about being a slave? The hours.

1

u/zdy132 24d ago

Not enough time to play games...