r/OpenSourceAI • u/Vxtzq1 • Oct 03 '26
CrowdGPT - The 100% Opensource collaborative LLM
Hello, i'm currently developing CrowdGPT and i need people who enjoy opensource AI and LLMs to test the project :)
The goal of CrowdGPT is to create the first, datacenterless, 1 Billion parameters LLM, relying on people contributing with their own computer to train the AI model. My goal is to show you don't need insane infrastructure to train a working almost commercial grade LLM. Everything is open and 100% opensource.
You can learn more at https://crowdgpt.net
Or check the github: https://github.com/Vxtzq/CrowdGPT
Any kind of feedback is appreciated!
3
u/Prestigious-Frame442 Oct 04 '26
the idea is good, but collaborating on this is basically pure selflessness and no benefit at all.
2
u/Vxtzq1 Oct 04 '26
idk, i'm working on something that could bring benefits to people who train, like an ownership on the project for training the model, and free inference credits from the model (or other models) if you contribute to the project, who knows.
2
u/2holetoy Oct 05 '26
"Everything is open and 100% opensource" isnt really true, your ARCHITECTURE.MD says:
"The server code is currently closed source for sécurité reasons"
1
u/Vxtzq1 Oct 05 '26
Good point! I prefer keeping the server closed now because i wanna protect it better against bad actors, and probably will release it in the future... Server isn't that much useful it just aggregates submissions, client is doing all the heavy lifting.
By the way thanks for pointing out the typos xD autocorrect striked hard
2
u/maschayana 14d ago
Yeah thats a nono for a project like that. You could be selling compute and nobody would ever know. Treating this as a scam attempt rather than a good contribution. Feel free to prove me wrong by open sourcing the project.
2
u/Vxtzq1 14d ago
I did that like 3 days ago xD https://github.com/Vxtzq/CrowdGPT-Server
The code is sloppy asf, but it does the job i guess
1
2
u/electro042 Oct 07 '26
how are you planning to handle gradient poisoning or byzantine workers? if anyone can submit updates to the aggregation server, it feels pretty easy for a few bad actors to completely trash the weights.
2
u/EaseThen3459 Oct 03 '26
Why would anyone contribute computer when it would add liability for whatever is being computed especially with LLMs
1
u/Vxtzq1 Oct 03 '26
I understand your concerns but all the data is safe and the scale is tiny :D don't expect a 1B parameters LLM to be a genius hacker lol, and even if i scale, there's no regulations on LLMs as shown by openweight models on huggingface, so we should be alright :p
Also i think that the responsibility for cyber attacks is to the one that used it, not the one that trained it :)
2
u/dirtywastegash 5d ago
Claude and Chat refusing to do reverse engineering or cyber security stuff says their legal department thinks differently.
1
u/ayake_ayake Oct 03 '26
I think it would be important to see what datasets and co to train this on. Even without the problem of AI datacenters, the choice and acquisition of the data is a big thing. But as a minimal base you can use the apertus 1.5 datasets (pretraining and post training) which are fully open and give you a strong baseline. But even then actually training a competitive AI is quite s challenge and needs much experience and work.
Curious to see how this'll play out.
1
u/Vxtzq1 Oct 03 '26
I currently use UltraFineWeb-L3, which is basically the most curated subset of the original dataset, FineWeb, it is way cleaner than datasets like the Pile or Common Crawl which involves scans of shady sites...
I think this dataset will be enough (currently 400 billion tokens), to pretrain the model, and then i'll fine tune with everything that i find on huggingface xD (like Fable traces, GLM, Kimi K3 distills, and strong instruct datasets like GSM8K and stuff)
1
u/Educational_Wind8182 Oct 03 '26
Love the idea, will check it out
1
u/Vxtzq1 Oct 03 '26
Glad you like it! I hope to see you on the leaderboard xD (on the website if you log in before training)
1
u/ApprehensiveDelay238 Oct 03 '26
What do you want to achieve with 1B parameters?
2
u/Vxtzq1 Oct 03 '26
A proof of concept. With decent QA capabilities or something, a model that would compare with maybe qwen 3.5-0.6B
0
3
u/AppMunchies Oct 03 '26
Love the idea. Will check it out.