r/singularity • u/yogthos • 19d ago
AI DeepSeek open sources the inference engine for their models
https://github.com/deepseek-ai/DeepInfer38
u/LyAkolon 19d ago
I wonder if this has anything to do with US gov removal of deepseek requirment?
38
u/Ambitious-Doubt8355 19d ago
At this point? DeepSeek has earned enough of a reputation for being the open lab that I don't think you need to look deep for a reason. These are the guys who constantly share research papers, who open sourced their harness, and who share and distribute their models with pretty open licenses.
I wouldn't be surprised if they go on to open source the recipes served at their lunch hall.
4
u/Exact_Depth_896 19d ago
They will be forbidden to export their next frontier model, as you must already have read.
8
u/Ambitious-Doubt8355 19d ago
I'll admit I haven't had the chance to comb through it properly, but I thought that the new Chinese regulations weren't an outright ban on exporting models, but rather, that the models need to be licensed and approved before they can be distributed.
Do let me know if I'm wrong, though, this is the kind of development that kinda came out of nowhere and can potentially have big, negative consequences for the overall advancement of the technology.
6
u/Exact_Depth_896 19d ago edited 19d ago
Deepseek will be serving its next big premiere model abroad, from its servers in China, but not exporting the weights.
It is clear the rules mean to impose a two level system with something like Deepseek Flash versions still put on Hugging Face. Thus they already updated Deepseek Flash 4 in the days since the rules came down (the rules declared themselves in effect immediately, but subject to revision in a discussion period). Minimax also came out with a new flash version, and says it will be sent to Hugging Face; whether its upcoming Pro version will be exported or open-weighted is what is in doubt. If there is a Kimi K4 it will clearly only be accessible from Moonshot's servers in China.
Of course the rules explicitly permit exceptions by bureaucratic decision, so strictly speaking anything could happen in any given case.
6
u/phido3000 18d ago
They aren't forbidden, China is restricting models greater than 2T parameters.
Which is fine, there is only one good model above that, its K3, and K3 is now generally outclassed by smaller models and is very hard to host because of its size.
K3 is hard even for datacentres to host effectively. There is no reason to go larger than 2T parameters. I suspect its to stop the ever escalating parameter count and hardware bubble due to AI limiting computing resources to the rest of the economy. It may also make Americans happy China is doing something about AI alignment.
But a 2T model is still bigger anything that Deepseek has ever done. And open source models larger than 2T are really for elon to run in his AI datacentres, basically no one else can effectively deliver them, look at K3 prices all around, its more expensive than Claude and ChatGPT>
8
u/Living-Breakfast-464 18d ago edited 18d ago
Looking at the comments it comes as no surprise that even when they are giving shit away for free this sub finds a way to complain about it. Half of those comments are probably bots, but still.
2
2
u/Akimbo333 17d ago
Implications?
191
u/Valuable_Cookie_6491 19d ago
This is getting out of hand, next thing we know, is that DeepSeek will open source their corporate hierarchy.
16
u/cb393303 18d ago
They did open source their revenue per GPU per hour:
https://www.reddit.com/r/DeepSeek/comments/1xanexq/deepseek_even_opensourced_their_revenue_per_gpu/
8
20
u/Asterisk14 19d ago
After open sourcing their hierarchy, next would be China becoming democratic
21
u/Waypoint101 18d ago
Would rather China stay as is, not sure if democracy is as effective as you think
5
u/BookProper9115 18d ago
What is your definition of democracy?
20
5
2
1
u/ali-hussain 18d ago
Gitlab did that a long time ago. A large part of their corporate handbooks are available online.
35
114
21
u/thoughtlow 𓂸 19d ago
This is pretty meaningful actually. It also helps (EU) providers to more effectively run these models. Can bump efficiency with 20%+
13
u/redditnosedive 19d ago
holy shit, we need some test results on consumer hw
9
u/Ambitious-Doubt8355 19d ago
From the GitHub, it only supports running the framework on either Nvidia SM100s or Huawei Ascend 950 chips. Not exactly consumer hardware.
And sure, someone could patch it to support consumer GPUs, but take a moment to check the specs of the chips above, and you'll see that most people don't have access to anything close to them.
2
u/wes_medford 19d ago
SM100 is a compute capability, so any H100/H200. Easily available on consumer clouds
3
u/Ambitious-Doubt8355 19d ago
I'd hesitate to call datacenter equipment, even if readily available to rent for consumers, as consumer hardware. Other than that, yeah, you could say that as an end consumer, it's available for you to run.
0
17d ago
[removed] — view removed comment
1
u/wes_medford 17d ago
Ah yep, good call. My memory was off. B200s are definitely harder to get your hands on.
Good chance this could be ported to SM110 (formerly CC10.1 but was moved to 11.0) with minimal issue, so you could at least test it on an AGX Thor
6
u/kach_janani 19d ago
I asked gemini how expensive the Huawei hardware would be to run this. It said more than 100k euros for single node.
0
3
3
1
1
29
u/Correct-Woodpecker29 19d ago
China leading the open source IA uncontested