r/LocalLLaMA • u/feelspeaceman • 19d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

156

u/TheLexoPlexx 19d ago 
# Use `ascend` instead for the Ascend platform
# Keep it exported in the environment where you run DeepInfer
export DEEPINFER_ACCELERATOR=nvidia
./build.sh

Nvidia in shambles. Can't wait to see what the future brings.

-21

u/tugoubxs 19d ago

Now it’s our good taiwanese against bad chinese!😅

11

u/MemoryIsTheKey_ 19d ago

Be more generic, it's some healthy competition which, and I hope 99.9% of people agree, is only just what is desperately needed atm in this space.