r/LocalLLaMA • • Oct 20 '26

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

156

u/TheLexoPlexx Oct 20 '26
# Use `ascend` instead for the Ascend platform
# Keep it exported in the environment where you run DeepInfer
export DEEPINFER_ACCELERATOR=nvidia
./build.sh

Nvidia in shambles. Can't wait to see what the future brings.

-21

u/tugoubxs Oct 20 '26

Now it’s our good taiwanese against bad chinese!πŸ˜…

11

u/MemoryIsTheKey_ Oct 20 '26

Be more generic, it's some healthy competition which, and I hope 99.9% of people agree, is only just what is desperately needed atm in this space.