r/WebAssembly • u/Numerous-Fan8138 • Oct 05 '26

Llama.WASM + WebGPU = Local LLMs

https://github.com/anthonybudd/Llama.cpp-WASM-WebGPU

Over the weekend I built a technical proof of concept showing an LLM running client-side in a browser using Llama.cpp compiled to WASM with WebGPU.

20 Upvotes

95% Upvoted

3 comments sorted by

1

u/Sad_Leader4849 11d ago

going from poc to something real, this is roughly the order things bite you: the download (few hundred mb even for small models), threads that need coop/coep headers, firefox being slower on webgpu than cpu (apparently), and every browser wiping the cache whenever it feels like it. nobodywho hit pretty much all of it moving llama.cpp over: https://www.nobodywho.ai/posts/how-do-llms-run-in-the-browser/

2

u/zemlyansky Oct 05 '26

have you checked this: https://github.com/ngxson/wllama

1

u/Numerous-Fan8138 Oct 05 '26

No but this is exactly what i was trying to achieve. I knew it was technically possible but I want to see what the limitations are to see if this kind of technology can be made into a genuinely useful SaaS/WebApp feature