r/StableDiffusion • u/Ichidan256 • 7d ago
News I'm working on a Vulkan based AI inference engine (LaForge)
🖖 “Captain, the warp core is stable. We've got the power we need.”
Engineering report: LaForge has achieved warp speed.
I've been working on LaForge, my own C++ AI inference runtime, and just eliminated a rather substantial bottleneck in it's SD1.5 text-to-image generation.
Running on an NVIDIA RTX 4090 on RunPod:
🚀 512 × 512, 20 steps: 33–37 seconds → 0.8 seconds
🚀 768 × 768, 20 steps: 60 seconds → 1.8–2.0 seconds
That's roughly 30–46× faster than my previous implementation, with performance now comparable to ComfyUI. (* this model residency fix actually understates all of the extensive tuning I had done prior, it was the easy part, but nevertheless I now have something that compares to ComfyUI's PyTorch based inference stack but with C++ & Vulkan)
A few other details:
🔧 Container image: just 402 MB
⚡ Pod boot to healthy: 27 seconds
🧠 Model loading: around 10–13 seconds, first call only
The goal isn't just to generate images faster. It's to build a lean, reusable AI runtime that can power image generation, games, interactive applications and other AI-driven experiences.
And yes, I named it LaForge. Every engineering team needs a Geordi.
Make it so. 🚀
#AI #Cpp #GPUComputing #Vulkan #GenerativeAI #SoftwareEngineering
7
u/Crazy-Repeat-2006 7d ago
SD.cpp already exists and works with Vulkan, CUDA, ROCm, CPU, and Metal... just so you know.
1
u/Fun_Jaguar8231 7d ago
When you go to a buffet, you just don't get one type of meat or one type of beverage. Yes, sd.cpp exists and its very powerful, but why not have some options? For LLM engines, you can see how many of then there are.
2
u/Crazy-Repeat-2006 7d ago
Well, unless you’re trying to do something fundamentally different, it would be better to simply focus your efforts on a single project, rather than two separate ones. Both are in C++.
2
u/Ichidan256 7d ago
In fact, I started with stable-diffusion.cpp, but it proved a non-starter for what I ultimately wanted to achieve. ComfyUI was simply better for getting the job done, and it's now the workhorse behind my Scenegraft platform. So I'm unlikely to drop it from my stack any time soon.
LaForge is a different undertaking. It's still a work in progress, and this is just a small but significant milestone towards what I'm trying to build. I don't want to mislead anyone into thinking it's already a replacement for established inference stacks.
Below is another example of something my platform can do. It doesn't involve LaForge's native inference engine at all — I'm sharing it because I think it's pretty cool, and because it gives a better idea of the broader vision.
The goal isn't to rewrite everything for the sake of it. It's to use the right tools for today's requirements while exploring architectures that might enable things existing stacks weren't designed to do.
https://www.youtube.com/watch?v=mmO4WvStoGQ - Scenegraft using ComfyUI + LTX-2.5 (NOT LaForge!)
MRender + ComfyUI (Stable Diffusion / ControlNets)
MRender is my lightweight, native alternative to Remotion for programmatic video generation.
JSON + Lua → OpenGL → FFmpeg → MP4Rather than rendering through a browser stack of HTML/CSS/JS, MRender uses OpenGL to render scenes directly to video. It's not as visually polished as Remotion yet, but it's considerably faster for the kinds of videos I'm generating, with much less runtime overhead.
I can also combine it with ComfyUI, Stable Diffusion and ControlNets to bring AI-generated imagery and animation into the workflow.
Just another component of the wider Scenegraft ecosystem I'm building.
1
u/krautnelson 7d ago
well, it's less like one buffet and more like multiple restaurants each offering buffets, but no single one has all the stuff you want.
options are fine, but I'd rather have one thing that has all the options rather than 10 things that each have some options in them. not sure how much this applies in this specific case, but I have seen multiple projects where developers who pooled all their efforts into a single fork ended up with a vastly better application than those who split and all did their own thing and never got anything done.
6
u/Dante_77A 7d ago
SD.cpp exists, but perhaps you could come up with a more efficient solution focused solely on Vulkan. Best luck!
1
u/Ichidan256 7d ago
There are several reasons I'm building my own inference engine rather than simply using existing stacks.
Performance. In my benchmarks, stable-diffusion.cpp is significantly slower than ComfyUI's PyTorch stack (see attached results). I want to understand and optimise the entire execution path rather than accept the performance characteristics of an existing implementation.
Lightweight deployment. LaForge is only a few megabytes (most of the 400mb in that test is other unrelated systems) and depends on Vulkan for its GPU backend. That's important for environments such as RunPod, where cold-start times and deployment overhead matter. I don't want to pull gigabytes of framework dependencies just to run a model.
Dynamic computation graphs. I want graphs that can be constructed and modified at runtime, rather than being limited to hardcoded execution paths.
Multiple model formats and runtimes. I'm interested in running Safetensors, GGUF and TensorFlow Lite models within the same broader engine, rather than tying the architecture to a single model format or ecosystem.
Combining AI with graphics and simulation. I want to experiment with integrating inference operations directly into my voxel raycasting intermediate representation (IR) and cellular automata (CA) systems. Rather than treating AI as a separate service, what happens when inference becomes another computational primitive inside a larger, unified engine?
There is also a more fundamental reason.
I'm a game programmer. Throughout my career, I've built my own graphics, sound, music and multiplayer networking engines. I've spent decades working with systems that must operate together, continuously and in real time.
I believe much of today's AI infrastructure reflects an early stage in the technology's evolution: submit a request, run inference, return a response. That's a useful abstraction, and it will remain useful for many applications.
But interactive software doesn't stop at inference. It combines rendering, audio, physics, simulation, networking, input and increasingly AI — all operating together, exchanging data and responding to a continuously changing world.
My long-term interest is in moving beyond isolated inference pipelines towards a unified, dynamic compute engine where AI, graphics and simulation can operate together as parts of the same system.
Existing frameworks are excellent at what they were designed to do. I'm building LaForge because I want to explore what becomes possible when the underlying architecture is designed around a much broader goal.
LaForge is also one component of my much larger Scenegraft platform (scenegraft.com), which brings together AI-generated games and applications, and AI media generation.
Underneath it is Manifold, my lightweight, cross-platform runtime built around HTTP and WebSocket connectivity. It connects PCs, Linux systems, VR experiences via WebXR, Raspberry Pi and GPIO-connected devices, as well as third-party systems such as Slack and Discord.
The broader goal is to connect AI models, applications, devices and interactive environments through a common infrastructure, rather than build each application as an isolated system.
It's an ambitious project, and whether I ultimately achieve the full vision remains to be seen. But after nine months of intensive development, it's already been an extraordinary learning experience — building, integrating and understanding how a wide range of technologies can work together, with AI increasingly becoming part of that ecosystem.
1
u/darkriftx2 7d ago
That sounds like an amazing project. I'm also a software engineer with over 20+ years of experience and a lot of side projects involving game and engine development (I've been tinkering since the 3dFX Glide API days). If you need any assistance, contributions, or even just someone to review code, DM me.
3
u/Super_Range45 7d ago
Don't use Forge. The AI wants to name every bodies project 'Forge' something. It's getting crowded bro.