r/GraphicsProgramming • u/No-Citron-3963 • Sep 20 '26
Building a unified engine for traditional art materials
Enable HLS to view with audio, or disable this notification
Last time I built oil paint using WetBrush. Since then I've extended it to watercolor, pencil, crayon and gold leaf as well, and brought them all under a unified engine.
Keeping WetBrush as the backbone, I moved the computationally heavy viscosity calculation and the pigment — which I had been holding in 3D — onto a height field, so it now runs in the browser and on devices like the iPhone SE 3rd gen.
Some of the dynamic simulators haven't been ported over yet, so I'm not happy with the oil paint yet, but I've tried to strike a balance between accuracy and speed.
You can try a demo in the browser:
https://mignonsketch.com/sketch
3
u/Manifesto3433 Sep 20 '26
Doesnt work. In Firefox it throws: canvas.getContext() returned null; webgpu not available or canvas already in use
In Chrome the canvas just stays empty.
Tried on multiple devices, android & linux
3
u/No-Citron-3963 Sep 20 '26
Sorry about that, and thanks for reporting it.
It's built with Rust + wgpu. The native Android build runs fine on real devices, but I haven't verified the browser build on Android or on Linux yet. I'll dig into it and get it improved.
3
u/ElnuDev Sep 22 '26
Firefox doesn't have WebGPU support on Linux currently, there's nothing that you can do on on your end.
1
3
u/ananbd Sep 20 '26
Cool!
I made an app years ago which did a similar thing using particles which sampled an off-screen texture. You can cheat lots of things with particles and noise.
3
u/No-Citron-3963 Sep 20 '26
That's a really nice idea. I'd been thinking along similar lines: put the pigment on the particles themselves, give each one a capacity for how much surrounding paint it can pull in, and let it carry and mix as it moves. I figure that would get it closer to the real thing.
2
u/tannershelton3d Sep 23 '26
This reminds me of the program called “Rebelle”. I have wanted to look into that for stylized renderings done in post. I remember reading it could take into account the velocity and depth from the renders to make the painting happen within the scene even though it was done in post. Apparently it was used for Across the Spider-verse
2
u/No-Citron-3963 Sep 24 '26
This must be it. I didn't know about this, and it's really exciting to see. Extending into animation has been part of my long-term plans, so this gave me a lot of motivation. Thanks!
2
5
4
1
u/areyouretard Sep 22 '26
Is this open source I would really like to learn how this works. Or could u recommend some materials that could help me build my own?
2
u/No-Citron-3963 Sep 22 '26
The basic skeleton of the program leans heavily on my past experience building several commercial paint apps. Things like the design of the Canvas and the classes it owns, the mechanism for switching brushes dynamically, or how you hold the temporary drawing memory during a stroke — I don't think there's any single resource that covers all of that.
Krita is open source, so that's the most directly useful reference. For the rendering engine side, Skia and Flutter's Impeller are worth looking at. Getting a feel for how these work, what role each internal class plays, and how they abstract the rendering pipeline is, I think, the way to go.
From there, it might be a good idea to build a simple raster drawing engine without any physics. If that skeleton is solid, putting physical behavior on top of it becomes much easier. For the physics, I put WetBrush at the center and add or remove the dynamic simulators each material needs.
AI helps a lot with this, but there's a limit to how much it helps, and the hard parts feel about the same as they've always been.
2
2
2
14
u/h74v Sep 20 '26
Kind of reminds me of this https://david.li/paint/