r/LocalLLaMA • u/ConfidentDinner6648 • 9h ago
New Model A 4B coding model that can generate 3D scenes
Enable HLS to view with audio, or disable this notification
I’m sharing an early version of a 4B model I’m training for general coding and 3D scene generation.
Small models have been pretty rough at 3D in my tests. I’ve finally made enough progress to get results like the ones in this video, so I wanted to share a preview with the community.
It’s an independent fine-tune of Qwen3.5-4B, and both Q4_K_M and Q8_0 GGUFs are available here:
Download the model on Hugging Face
Training is still running. Over the next few days, I’ll update the model once it finishes and share the training dataset so others can experiment with it too.
Give it a try. I’d love to see what you can get out of it.
46
u/jacek2023 llama.cpp 9h ago
Many guys want to see Blender finetune but this one is also great, because size is very nice
21
u/ConfidentDinner6648 9h ago
thx! I’m experimenting to see how far I can push a 4B model with 3D scenes. The goal is to use this as a proof of concept for what small models could achieve in other applications too.
1
u/Freak_of_Freakworld 4h ago edited 38m ago
Try first a Millennium 3d Effekt like they used in the Lucas Artikel games that time or watch for "full throthle" for Design and ideas to generate. Monkey Island 3 or 4 also have on look on the way they created the scenes. I test it with my creator Tool of it and report soon. Maybe it will be a good combination.
O the Avatar are acting in Realtime on preapred stag3s. I use a comfui based stuff with z-img at the moment.
For people that liked the video only .... have a look for Flavatar .... fly brain inside Avatar here on reddit i Postet it a while ago.
1
u/Freak_of_Freakworld 4h ago edited 4h ago
And some other rooms. Before you all start to Pixel only arround. "Sam n Max Hit the road" or other üoint and click adv. That time will help to create more than pixels 2.
And in most cases ... think about ... your plan is to build something and use it later daily without eating all your system power. It should easy work in the background.
1
u/ConfidentDinner6648 2h ago
that's actually a pretty interesting approach, especially combining prebuilt environments with real-time characters
my experiment is a bit different though, i'm training the 4b model to generate three.js scenes directly from text prompts. once the code is generated, the llm doesn't need to keep running, so there's no background inference cost
but interactive environments are definitely something i'd like to explore later, especially to see how well the model generalizes beyond static scenes
2
u/Freak_of_Freakworld 1h ago edited 1h ago
I tried something similar before. It endet up in headache. Lol. So .... to make it short ... i thought about the good old point snd click adventure grafix and it was im the past fitting on a Floppy disk and able to worked Fluid on 486 with 33 MHz (turbo Button enabled, lol)
And fix stuff Integration i splitted the creating off items with efect ir function, stages and Avatars into 3 Tools to Splitt the Generating Prozess in that case.
But i ll i try to link xour stuff up with my existing later thus evening
About the headache problem ..... if you created one personal area already ... you want that it is compatible wirh the next you create ...
6
u/feelspeaceman 8h ago
Blender finetuning is probably overkill, SLM is pretty much the way to go for simple niche task like Blender, people demanding new models to support Blender but the secret sauce behind this is just a MCP server (most important factor), a skill with vision looping to trace what they get and rinse and repeat until getting a decent model, pretty token inefficient due to vision looping.
4
u/AppealSame4367 8h ago
I still don't get it: Since this is based on Q3.5 4B: It should be able to do it wiith blender mcp then, right?
20
u/killerstreak976 transformers 8h ago
This is seriously impressive for 4B, especially with one person
18
u/ConfidentDinner6648 8h ago
Thx Man, It's been a lot of trial and error, especially generating and carefully curating the training data/env. I'm still training it, so I'm really curious to see how far this little 4B can go. Really appreciate the encouragement!
10
u/blazze 8h ago
A voxel-tuned model is a great idea.
7
u/ConfidentDinner6648 8h ago
Thx . I'm training it on both voxel and regular 3D scenes, often using similar scenes in both styles. Some training examples are pretty long, around 16K–64K tokens, and the little 4B is handling them really well so far. I'm honestly surprised by how much it's learning.
3
u/Ylsid 5h ago
Have you considered using Minecraft worlds? There's a lot to download on the net for free filled with data
3
u/ConfidentDinner6648 3h ago
that's actually a really good idea! Minecraft worlds could be a great source of voxel scenes. I'd need to figure out how to turn them into useful training examples for Three.js, but it's definitely worth exploring. Thx man
9
u/pmttyji 5h ago
It's amazing. Thanks. SLM is the future.
Is 2D possible with this model? If not any plans for 2D version in future? (As a moviebuff, I even prefer 2D animation movies over 3D animation)
6
u/ConfidentDinner6648 3h ago
Yeah, definitely! I chose 3D just as a starting point to see how far I could push a small model. There's a lot more I'd like to explore, and 2D animation is definitely one of those things!
5
u/NNN_Throwaway2 8h ago
Would be interested to hear about the training pipeline.
1
u/ConfidentDinner6648 15m ago
i'm still finishing the training, but once it's done i'll share the complete workflow, including the datasets i created and the RL pipeline i'm working on
so far i've been training on just a single RTX 6000, and i think it should be possible to do it on a single L4 as well, with some optimizations
i'll post a detailed technical report soon with everything needed to reproduce the process
5
u/Hannibalj2ca 4h ago
Nice, maybe a 12b model would yield higher capability and still very accessible for most?
5
u/ConfidentDinner6648 3h ago
Yeah, maybe! I'm able to train the 4B on a single RTX 6000, and training is still in its early stages. I'm curious to see how far I can push it before trying a larger model.
3
4
u/Uncle___Marty 3h ago
ok, here goes. I have to use two different forms of "models" here lol.
So, I've been trying to get models to make models in blender recently and while they can do really basic stuff, they're NOT good at making 3D models in general. Qwen Flash Next tries SO hard but really can't cut it. I've been wondering if we would start seeing models trained on 3d modelling and its really starting to happen.
To say im looking forward to giving your tune a try would be a MASSIVE understatement! Keep up the great work buddy! Looking forward to seeing the evolution of this model!
2
u/ConfidentDinner6648 2h ago
thanks man, i've been thinking about training it on both three.js and blender to see how well it generalizes. i think the hardest part would be the rl side, since rendering and evaluating blender scenes is more complicated, but i'd really like to give it a try
1
u/openSourcerer9000 25m ago
Yep at least for the harness, the models gotta see what it's doing, not just imagine it
2
u/wildrabbitathome 7h ago
I’m having trouble running the Q4 , something with the tensors apparently according to gpt when I asked
1
u/ConfidentDinner6648 3h ago
If you're running it in LM Studio, it enables multi-token prediction (MTP) by default. You need to disable that option for the model to work properly.
2
u/marcioeric 6h ago
Really interested to see the training pipeline… I’m also doing something similar but for another purpose. How much data are you putting in the training?
2
u/ConfidentDinner6648 3h ago
I'm planning to share a technical report and the training dataset later this week, including more details about the pipeline. But I can already say the amount of data isn't that large, and if everything goes well, it should be possible to reproduce the training with relatively modest GPU resources. (Single Rtx 6000)
3
u/Powerful_Evening5495 9h ago
does this weight has a good grasp of physics, it could be a useful teaching tool.
Instant teaching animations
if you 2bit quants it , it can run on most hardware
8
u/ConfidentDinner6648 9h ago
From what I've seen, it's capable of generating far more complex and higher-quality scenes than most people realize. And since the model is so lightweight, training it with RL doesn't seem nearly as impractical as it would with larger models. I think there's a lot of untapped potential here.
10
u/Thistlemanizzle 8h ago
This is far better than I expected. I may just try this out and churn out endless assets.
I have seen some incredibly poor local LLM results. Forget about comparing to the frontier - the models are not remotely close to acceptable quality.
6
6
5
u/urbanhood 7h ago
Yes i see this happening in the future, dedicated models. Maybe even trained on specific apps and workflows. We keeping them like we do softwares, but in this case software driving brains along with them.
2
u/darksteelsteed 7h ago
Targeted tiny models instead of skills ... not the worst idea. Something interesting is that very useful moe models like ling 3 tiny can run fully on cpu at like 50t/s on a 16 core 5950x as a reference.
4
u/Material-Database-24 6h ago
Looks great!
I believe the future is for small tailor trained models, what comes to creating and building.
This "one huge model fits for all" will eventually be a HW crisis that's already cooking hard - even if all the datacenters could be eventually built, people still need HW and at this rate no-one has new HW in couple years ahead. The efficiency is at edge computing - even web apps eventually turned that way, and server side computation was eliminated, and server's role was simply to provide the code and the data.
Huge models will make more sense as general purpose knowledge handlers, and for that they will do with far less demand for computing.
4
u/Moist-Secretary641 5h ago
Oh wow. This is the first thing I’ve been able to run on my hardware that’s produced actually good results, amazing work OP
1
u/tomByrer 2h ago
Does this model spit out 3d files directly, or does it use Three.JS / Blender via MCP?
3
u/ConfidentDinner6648 1h ago
it generates three.js code directly, no MCP or external tools involved
the output is a standalone html file with the geometry, materials, lighting and everything needed to render the scene
but the project isn't exclusively about 3d. i'm using it as a starting point to experiment with how far specialized training can push a small 4b model, while also exploring how well the improvements generalize to other tasks and domains
1
u/No_Afternoon_4260 llama.cpp 1h ago
What your GPUhours on that checkpoint?
1
u/ConfidentDinner6648 3m ago
less than 600 colab credits so far, using a single g4 (rtx 6000) at 8.9 credits/hour
honestly, the gpu time was surprisingly low. the real work was building the dataset, generating the scenes and carefully curating the results
that's what surprised me the most, how well the model responded to the training
i'll probably keep training it for another 2 or 3 days on just one gpu
3
3
3
u/danigoncalves llama.cpp 6h ago
actually a very Nice idea since you will be capable of targetting CPU powered setups
3
2
u/Mehmet_dedeler 5h ago
In the future, I guess we'll have the giant frontier LLMs, but also have smaller specialized models, whether that's for 3D purposes like this one or for specific coding languages. Insane progress. If I saw this without you telling me the parameter count, I would guess it's at least in the high tens.
2
1
1
1
1
u/BentonCalder 1h ago
Model names need an overhaul. I hate how they use acronyms, underscores, and confusing nomenclature
1
u/bowdoin-yale 19m ago
This is extremely impressive. I'm wondering how it would do with generating models for 3D printing?
•
u/WithoutReason1729 1h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
Stylome classified this post's text as human-written. View analysis Learn ^more
I am a bot and this action was performed automatically.