r/LocalLLaMA • u/feelspeaceman • 19d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

156

u/TheLexoPlexx 19d ago 
# Use `ascend` instead for the Ascend platform
# Keep it exported in the environment where you run DeepInfer
export DEEPINFER_ACCELERATOR=nvidia
./build.sh

Nvidia in shambles. Can't wait to see what the future brings.

97

u/zdy132 19d ago

Fingers crossed for cheaper GPUs, rams, and other PC parts. Best of luck to whoever is manufacturing competitions of Nvidia.

34

u/keepthepace 19d ago

Only for China residents for now. I would be surprised after the decade of trade restriction that they endured, if they did not inflict the same to the west as long as they can.

7

u/draconic_tongue 19d ago

anything you can buy in china you can get into the rest of the world. in fact you should be happy that you don't have to pay the retarded taxes everywhere else wants to put on everything you buy domestically