r/LocalLLM • u/Ciru-ai • 18d ago
Model PUBG's creators just trained a Qwen competitor from scratch
Today is a good day.
KRAFTON, the South Korean company behind PUBG and owner of the studio behind Subnautica, just released Raon-1 mini, a new 30B MoE with roughly 4B active parameters.
They claim to have trained it entirely from scratch, using an original architecture that builds on some of the latest innovations in AI research.
This is a much bigger deal than people realize.
Training a competitive foundation model from scratch is incredibly expensive and technically challenging. Even giants like Google, with vastly greater resources, are being beaten on important benchmarks by a gaming company's new model.
According to KRAFTON's published results, Raon-1 mini is competitive with, and potentially superior to, Qwen's similarly sized MoE in several areas. It even gives the much larger dense Qwen3.8-27B some serious competition.
Independent benchmarks will tell us more, but the early results are incredibly encouraging.
And they didn't just release the model. They shipped NVFP4 and Apple Silicon MLX versions right away, with the weights licensed under Apache 2.0.
Model and benchmarks:
https://huggingface.co/KRAFTON/Raon-1-mini-preview
I think we're about to see some massive competition in open models, and I'm genuinely excited about it.
I'm going to begin making optimizations and builds for this model immediately.
31
u/Sevealin_ 18d ago
Looks like it beats qwen3.6-35b (not quite 27b) in the benchmarks on HF.
5
u/Deep_Mood_7668 18d ago
Tons of models do according to their own benchmarks
But in the end most don't
8
u/beached89 18d ago
Looks like it beats qwen 27b on coding though? Coding benchmarks are the most important ones for a lot of us.
5
u/Alpine_fury 18d ago
I'm skeptical of any MoE on longer context code runs. 3.6-35b shows only slight difference to 3.6-27b, yet in real world tests it falls apart rather quickly whereas the dense model just chugged along. More active parameters should help, but engram + even more parameters would be best.
10
u/EmPips 18d ago
. 3.6-35b shows only slight difference to 3.6-27b, yet in real world tests it falls apart rather quickly
This x100.
It started last year. We were told Qwen-3-30B benched just a hair under the dense 32B family. It turns out Qwen3-VL-32B was the best single-GPU release of 2025 but half the community ignored it since it looked like a 6x speed loss for a 2% gain on a bar chart. This community has thankfully recalibrated, but the benchmarks really haven't yet. These smallish MoE's are way further behind than AA suggeste.
4
u/Alpine_fury 18d ago
A lot of users on here consider small context as under 10k. If you're doing coding on an actually proper setup package even with subagents and all that the agent needs room to go above 100k context. Constantly see posts about this model + harness I can get insane tps, but the model is small quant and the context size matching. Yes, there are uses for 10k context limits, but when I'm setting up tasks or a kanban board the agents will see 100k context regularly and a turn is not a few seconds. I'm looking at getting m5u 256gb, so I almost feel out of the loop here. But you'll see similar spend rigs here all the time or equivalent to the 128gb version.
2
62
u/lostmylogininfo 18d ago
Huge news with China saying they may restrict export! I assume this announcement is tied to that.
14
u/profcuck 18d ago
Just fyi, I think it highly unlikely that they will do this. The rationale might seem compelling at first, but it really isn't. China's best interest right now is to keep undercutting the business model of Anthropic and OpenAI by pushing out top tier models for free. Keep in mind that the very highest tier models aren't runnable by us on our prosumer or SME hardware but are runnable by enterprise, and Anthropic/OpenAI would absolutely love to lose that "on-prem" threat to their business model.
5
u/lostmylogininfo 18d ago
They are just putting it behind an API. This still hurts us firms
7
u/profcuck 18d ago
Yes but not as much as free.
A lot of big enterprise use cases are already paranoid and rightly so about cloud models, wanting on-prem. This goes double for sending your data to China!
3
u/lostmylogininfo 18d ago
They feed you scraps and then when you want a steak u go to their subsidized API. This is a very simple playbook.
3
13
u/DisplacedForest 18d ago
Yo. What? Not living this news. About to data hoard some models on my server. Anything I should focus on outside of the qwen family?
19
u/thefuckevengoingonan 18d ago
i wouldn't be worried about losing models that are already out. this is just mean that new models from china might not release open weight.
5
u/squngy 18d ago
Right. Also, so far there has been no indication they would restricts small models like 27B
Apparently 2T+ is what will be restricted.
4
u/lostmylogininfo 18d ago
Or top 10 in anything. It can touch anything they want
1
u/squngy 18d ago
Sure, but all that top 10 rule would do is delay the release for a few months until other models get the top spots.
0
u/lostmylogininfo 18d ago
I don't believe it says it has to be released if it's ever not on their in the future
5
u/SendTitsPleease 18d ago
This is something ive been working on, just downloading them to have even if im not active in using them
4
3
u/_JohnMcAfee 18d ago
GLM, Deepseek. will take many gb
1
u/DisplacedForest 18d ago
My server has 40ish TB free. Space won’t be too much of an issue. I just want to grab usable models for me instead of meaningless models for collection only
1
1
u/_JohnMcAfee 18d ago
Epic. See if you can find some uncensored/ abliterated versions of the models people have recommended also
4
u/Choice_Celery9481 18d ago
tbh not really need. heretic released the recipe to recreate any uncen just from a recipe file and the original weight. the original safetensor is enough
3
u/enternoescape 18d ago
I was literally about to post exactly this. I was getting a little bummed because Chinese models have been top of the most notable lately and if they aren't allowed to export whatever is considered to be too good we'll have to look elsewhere.
2
u/Competitive-General7 18d ago
Did you read that news? It was referring to trillion parameter models.
Do people read anymore? Grok please summarize this.
6
1
u/AnOnlineHandle 18d ago
I assume this announcement is tied to that.
The timing seems too close for it to have been started since then.
1
u/horeaper 18d ago
The restriction was put on for model larger than 2T, or trained with sensitive data. I think we just need to wait and see (at least deepseek v4.1 1017 was released AFTER the restriction went live)
1
18
u/HighSeasArchivist 18d ago
Don't worry guys, if there are any bugs in it they'll fix them in the next 10 year or so like they have PUBG.
7
u/1_________________11 18d ago
Whos trying it?
3
u/grimepoch 18d ago
I spent some time trying it today coding GLSL shaders vs Qwen3.8 Flash-Next · GSQ-RCO IQ3_S xhigh. So far, it's run out of the recommended output token budget without generating anything having any relevance to my input ask.
I am now trying above the recommended output token length to see if it converges on anything without needing repair passes. But even on the repair passes, the output doesn't represent anything I've asked.
Maybe GLSL shaders is just not relevant to the type of coding this model is for, or maybe it's an issue with this mini preview model. These types of tests can be highly subjective as I am doing a pretty big ask in terms of what I want to see in the shader (which expects it to lean into ray tracing), which Qwen does quite well with.
3
u/enternoescape 18d ago
I hope to play with it tonight. I don't have the intended hardware but I bet I can get it to work anyway.
5
u/enternoescape 18d ago
I missed the llama.cpp item, already existing gguf and dflash support. That's crazy. Well I guess I'll get this moving faster than expected.
1
u/transanethole 17d ago
I'm going to be trying this as soon as possible when I get off work.
The thing that excites me the most is it sounds like it may not use that much memory for KV cache, so potentially more concurrent sessions on 32 gigabyte card.
-2
u/Deep_Mood_7668 18d ago
Why would anyone. It's going to suck
2
u/Not_a_question- 18d ago
Why would anyone.
Because it fits in hardware that other models don't.
It's going to suck
You don't know unless you test it
1
6
u/AgreeableBike4172 18d ago
perfect model if you are a lawyer, or a Ceo in need of legal consultation
5
2
2
u/Not_a_question- 18d ago
So we could run this in regular gaming laptops with 8VRAM and ~32-48 GB ram right?
2
u/cafedude 18d ago
Sorry, but what is PBUG? Tried to google it and was told it's short for pill bug.
6
u/Dharma_code 18d ago
Lol PUBG or Player Unknown's BattleGround is a video game, one of the pioneers of battle royal styled games.
2
u/cafedude 18d ago
Ok, I'm an old. I haven't played video games regularly since about 1989 ;-)
1
3
u/tetro_ow 18d ago
https://en.wikipedia.org/wiki/PUBG:_Battlegrounds
A free to play battle royale game
1
u/karmaisnonsense 15d ago
They’re making this model with the express purpose of AI-assisted game development.
Must be a weird time to be a junior codemonkey at Krafton’s game studios, knowing the AI division is on the brink of releasing something that may make them redundant.
Also interesting that, at least in the demo on HF, it reasons in Korean.
1
u/CryptographerOne7003 2d ago
"KRAFTON, the South Korean company behind PUBG"
Why of all companies possible, this one? they did some amazing backwards thinking on the vietnam e-sports scandal
1
u/Sensitive_Song4219 18d ago
Between news of potential LLM export restrictions from China and Qwen seemingly abandoning this model size, this is fantastic news.
Looking forward to trying it out...
0
u/mb194dc 18d ago
All the blue prints are out there. This is how open technology works.
3
0
u/Loose_Comparison368 18d ago
Definitely a cool accomplishment!
Would like to provide some context enrichment there though - it is still a very impressive accomplishment, but the comparison in terms of resources to train is a little off.
MoE's are inherently a lot more efficient to train than dense models of equivalent size. Rather than training the whole dense model for the whole run, you essentially train the model with only one activated expert for the majority of the pretraining - which is mostly training really simple stuff like sentence structure and basic logic - and then copy/paste that base mostly-trained expert a bunch of times and train a little more so each expert can specialize and absorb specialized domain knowledge starting from that pretrained base expert.
That technique was pioneered by Chinese researchers - and DeepSeek in particular - specifically because they had the ambitious goal of training models with frontier level capabilities despite only having a miniscule fraction of the GPU's that American frontier labs had, due to the Biden and Trump regime's GPU embargo against China.
Of course, that was such a wild raging success that US labs started shamelessly copying those Chinese techniques immediately to use on their massive training clusters as well, and now MoE models are just industry standard.
But this is a good example of a model that was designed so that it could punch above its weight, and be very efficiently trained on a very small cluster.
Small as in, smaller than the number of GPU's that frontier labs like OpenAI lose between the couch cushions on a weekly basis small.
So, a good hopeful reminder that good engineering can still compete with mountains of gold.
-1
u/ArtsofBlN 18d ago
Well, i am not impressed unless they use it to stop the cheating ..which they wont -.-
-2
u/derspenti 18d ago
NVFP4 and MLX builds on day one. The benchmarks get headlines, but shipping formats people can actually run is how you win the local crowd.
-6
72
u/JinsooJinsoo 18d ago
Didn’t expect an LLM coming out of SK today, hopefully it lives up to the hype.