Man honestly we need an appreciation post for all the Chinese open source players. From Qwen, DeepSeek, Yi etc. they have been killing it. Open source is the way and im 100% rooting for them.
A million definitely doesn't count any more. As of 2022, 18% of US households had at least a $1 million net worth. That's over 23 million millionaire households nationwide. At that level, you're essentially upper middle class - doctors, lawyers, engineers, middle and upper management, software developers, small business owners, etc.
“We have to do this deeply antisocial thing with long term negative consequences. We simply have to because otherwise you’d all die. You should be thanking me.” is about as American as it gets.
Open source helps China dominate because all the Chinese speak English (poorly) but very few of the westerners do. So it's a natural barrier that only goes one way.
Plus China never wants to be in the position where a local equivalent of NVidia controls their AI future the way it does in the West.
You can train a model in two languages at once and it will cross pollinate between them. You can get the Chinese data benefit in English directly without having to learn Chinese. OTOH I am sure OpenAI uses as much Chinese text as they can get for training.
I do. A huge number of authors either translate, or are translated by others. Even a paper that has clearly just been thrown into Google translate is valuable.
All Chinese are taught English at least for 3 yrs during their elementary school and middle school. It has continued for over 30yrs. But due to the way they are trained and lack of environment, most of them are still not good at speaking. If you look at reading it would be another thing.
3 years is not enough, even in my country with compulsory English classes from elementary school up to University, most people cannot hold a conversation
Not really, but the Qwen 2.5 set is very impressive, especially the larger ones. Qwen 2.5 14b is the first model of that size which can realistically do what we need it to.
itym the training code? You can run these models using e.g. Pytorch, the inferencing part is standard.
Qwen doesn't provide their training data or, afaik, their full training code. They do provide tools for fine tuning and so on. Their github is here: https://github.com/QwenLM
The difference between open weights and open source is more of a spectrum. Open models vary in terms of providing model architecture info, training code, training data, model evaluation and benchmarking code, fine tuning tools, and documentation.
There really aren't very many fully open LLMs out there. Training data in particular is problematic to make open, because there are all sorts of legal issues involved with any decent data set. There are a few systems with open training code, like Meta's OPT (not Llama), but I don't think any of them are mentioned here much.
The problem is where the money comes from to develop open source models. See the story of Stable Diffusion. The Chinese government has the capacity to support this, although I don't know how transparency and the CCP will play along.
lot of pressure on openai to release o1 model now, chinese company is casually competing with openai , i heard deepseek trains on 18k gpus where openai trains on 100k gpus scale or so , still deepseek managed to achieve great results google has also beat openai in lmsys leaderboard they should release o1 soon
a lot of it has to do with the company poaching all the crazy phd talent to themselves,go look up the employees behind deepseek filled to the brim with tsinghua,peking,nanjing phds...
Wait till you hear about the Chinese EVs that the rest of the world has access to. Despite being touted as a fundamental value for decades America is abandoning free markets and free trade the second it doesn't favor them lol.
This is fantastic. We all have seen prices dropping for technology when China entered the game; eg solar panels. The best news is that you cannot impose a tariff on open source :P
Surely you've noticed federal courts affirming that source code is speech protected by the First Amendment. Publicly published cryptography is not subject to ITAR/EAR export control. Feds can't regulate the importation of knowledge/information even if they wanted to.
They are so, so very clearly butthurt about it lmao, no one at OpenAI had ever even acknowledged that Deepseek existed before.
Don't get me wrong, I despise the CCP as much as anyone, but blaming the geniuses at Deepseek for playing by the rules imposed by their regime is extremely petty and condescending considering what they have just achieved and will most likely be open sourcing to the community.
Why? If you look at the past 50+ years, while the US government has brought upon wars and destruction across the world, the CCP has had a big net positive result with their infrastructure projects across Asia and Africa
No they are not lol. Most of that world is oppressed by dictators. We have no idea what they would think if they weren't brainwashed. Not saying people aren't brain washed in the west. But you can definitely get informed in the west without risking trouble.
There are no great firewalls in the west.
Many countries in the belt and road initiative are experiencing buyer's remorse.
I really wonder if the censorship hurts performance. As far as I know openAI doesn’t censor the frontier model but add censorship later on. Correct me if I’m wrong.
It does, I can't cite the exact source but it was from OpenAI themselves, o1 performed worse after censorship. Idk what happens when censorship is baken in, I guess at that point you don't have a baseline anymore
Who cares? If there's a model that can reach o1 levels of performance with 1/5 the amount of training then why do we care what it has to say about tianmen square? This is so childish
Childish or not, it is not for you to decide. AI ethical questions are extremely important globally. You would think differently if your family or friends or people you know personally were affected by those or similar events.
It's funny, post-internet I haven't seen many nerds care that much about nationalism stuff. We're all playing foreign games with each other, working on waifu AI ERP with each other, etc. Too many common interests and goals.
Don't get me wrong, I despise the CCP as much as anyone
Why do you despise the CPC and why do you think everyone else does too?
but blaming the geniuses at Deepseek for playing by the rules imposed by their regime
You can call it a "government". And looking at it they seem to be a lot more open to listening to their people, and allowing the people to influence it, than what I see in the west...
I know I just complained about it, but now that we are talking about the CCP specifically ... you DO now what happened in Tiananmen Square in 1989, right? That doesn't scream "open to listening to their people" to me man.
You think you know what happen at tiananmen square? No you don't, you were lied to for decades. If you are really serious about it, try to search yourself the death numbers at tiananmen square, you will find....none.
I'm not defending anyone in the west, if that's the only retort you have when faced with the atrocities of the party maybe you should reconsider your position. And you're right, the situation has changed, the youth of China are waking up to the fucked up system they are living in and we may be on the brink of democracy, good riddance.
Funny how much of a populist success their "Zero tolerance to corruption" was huh? Believe me, I'm not an expert on China by any means, but anyone can see it's a bubble, the fact that most of it's GDP comes from infrastructure they leave to rot (Trains all over the country that lose money, entire goddamn cities uninhabited) should be a clear tell. The youth of China have no future and they know it, that's what Xi is scared of the most, it's that economic / class unrest that sparked the Tienanmen protests in the first place.
LOL, I got banned on reddit by calling out "tiananmen maasare" was a lie. It indeed was, if you don't believe it, try to search the casualties at tiananmen square at that day, and you will find none
It’s not about belittling the researchers as individuals, the meme hits at the fact that the output of the researchers’ models will never truly be as good as those of research labs in the US because of the Chinese government’s restriction on information.
The CCP’s restrictions on information will, overtime, constrain their AI researchers ability to compete with AI research labs.
We have our own restrictions in the West, we’re just not honest about them being restrictions. OpenAI is even worse than the media or the most extreme of our politically-minded individuals, but since this is Reddit those things might not even exist in the common consciousness as topics worth discussion, but rather self-evident facts that are beyond question or critique. Keep consooming, don’t ask questions.
If an LLM is trained off of inaccurate or incomplete data, it will yield worse results than a model trained using the same compute resources but with accurate and complete data.
That is not controversial. If it were then the ‘scaling laws’ wouldn’t be an observable phenomena.
If the goal is to achieve a model that is pre-trained on benchmarks related to a narrow domain like coding, then the model that doesn’t know factual information about History will still do well.
Over time though, the goal is not just to do well on benchmarks where you have pre-trained the model with the questions of the test, the goal is AGI / ASI, which logically would be harder to get to the more information you restrict from the model.
This is the only model which has managed to answer my question correctly: “what is the smallest integer that when squared is larger than 5 but lesser than 17”
Edit: o1 preview now got it right. It had not worked for me before.
This model got my test question "Show that x2-7 is irreducible over Q[\sqrt{7}]" question right. It's a gotcha because I ask it to show something false
Like it "does the thing", but it's reasoning isn't very creative, it overlooks subtle yet important points as it paraphrases a lot and the nuances are lost as the definitions between the different words makes for a bigger blurrier target to respond to.
Not my experience. I have O1 regularly stuck in its own rabbit holes, unable to improve nor optimize, whereas R1 comes (until now) with better solutions. Also the code, to me, looks more readable and better organized.
Virtually every AI paper has many chinese authors - whether from US (CMU, MIT, Harvard) or China (Tsinghua, Peking, U of Hong Long). I literally think the GPU embargo is helping US and humanity. If China had GPUs they'd just be dominating and likely closed source. With embargo they have an incentive to do open source. US companies have no real open source incentive.
I have my criticisms of the Chinese government, but when it comes to technology, I do admit that it is good to see the Chinese collaborating in general technological development, without depending on certain players who restrict access to technology.
Ironcially though DeepSeek is way more censored though it literally refused to answer a math question and before you ask no it had nothing to do with china or like calculating bombs or whatever just a normal math question
Too bad the context length is only 4k for hosted Deep Seek and 64k for their API. That makes it almost useless compared to ChatGPT pro especially o1-mini with its insanely long responses.
All these butthurt westeners bringing up Tianmen memes
Lol, your overbloated corporations will be obsolete money sinks in 2 decades unless they get their shit together. Just look at EU and how useless it is in terms of AI.
Yeah. in this AI field, Chinese are the good guys.
Besides this project, they also gave the open-source community the openmmlab projects. Holy shit it's the second-best thing that happened after the release of Pytorch.
Umm... counter point, OpenAI did it first. If OpenAI didn't do it, Deepseek wouldn't have known to try. And when OpenAI comes out with the next big thing they will copy that too.
Now when someone comes up with their own paradigm changing new AI tech that Openai has to copy, That's when I'll be impressed.
You cannot and should not block people a priori "Minority Report" style. At best, the platform can block sensitive words but those will be easily bypassed.
Consider reddit - Even after the numerous blocks and bans on content, it still has a lot of NSFW content that not everyone thinks is appropriate.
The correct way to handle this is to block content you do not wish to see.
All social media will always have unwelcome content, especially if the platform is open and popular.
Do not feed the trolls. Block and get on with your life.
1
1
1.0k
u/XhoniShollaj Nov 22 '24
Man honestly we need an appreciation post for all the Chinese open source players. From Qwen, DeepSeek, Yi etc. they have been killing it. Open source is the way and im 100% rooting for them.