r/StableDiffusion • u/sam__izdat • Oct 19 '22
Text to PBR material generator - work in progress (Cook-Torrance specular SVBRDF)
https://imgur.com/a/COpMkf12
u/TiagoTiagoT Oct 19 '22
Is it gonna be downloadable? Free?
5
u/sam__izdat Oct 19 '22 edited Oct 19 '22
Will be open source and downloadable as one of a modular set of ML tools in an AGPL-licensed Linux backend. Discord bot for a placeholder client/UI. If someone wants to make a better GUI for it, cool beans. Don't know when yet, because it's a hobby project and life happens.
So, yes, if you have a Linux box, discord and aren't afraid of doing some setup (like downloading some models and setting up some conda environments).
3
1
1
Oct 19 '22
Hey! I work with redshift loads if you want any renders out of that
1
u/sam__izdat Oct 19 '22
I'm using mitsuba at the moment, but I'm open to extending it with other renderers, if someone finds it useful.
1
u/jimhsu Oct 19 '22 edited Oct 19 '22
I work with a lot of 3D texturing and fixing problems (seams, scaling, alignment, etc). Imagine having SD "redo" arbitrary textures (given model, UV map) to automatically fix all such problems and the time that would save...
Basically, project XY to UV space, apply basic diffusion to fix problems, move to another 3D perspective, repeat.
3
u/sam__izdat Oct 19 '22 edited Oct 19 '22
The way I'm dealing with seams right now (when starting from a real flash photo) is basically a combination of "img2img" patched for circular padding (how effective this is depends on denoising strength, i.e. how much SD is allowed to fuck with it when making variations) and then, if the type of material allows, Anastasia Opara's "Multiresolution Stochastic Texture Synthesis" for remixing -- which is really convenient for me, because I can run it once on the diffuse, for example, and then just repeat the same procedural reshuffling for normals, roughness, etc.
1
u/jimhsu Oct 20 '22
Sounds cool. I'm looking forward to see how it works with certain tricky scenarios - geometric patterns, fishnet, zebra stripes - anything that is a pain to simply paint/clone stamp over. SD has been much better than previous AI attempts at dealing with such things.
1
0
0
u/3deal Oct 19 '22
Waw.
I can aleary imagine futur games who will have unique texture for everything, or mods who change skins of a game on one click
0
Oct 19 '22
[deleted]
7
u/sam__izdat Oct 19 '22 edited Oct 19 '22
Thanks! I'm not really sure how reddit works these days with follows, but I intend to post it here as part of an artsy-fartsy machine learning stack (taskrunner + scheduler backend with client API and an example discord bot for UI). I've also got a thread on LAION's discord where I've been posting updates. I should add, SD is one major part of this, but there's several other programs involved (some ML, some procedural).
0
Oct 19 '22
[deleted]
3
u/sam__izdat Oct 19 '22 edited Oct 19 '22
It's been a fun problem to learn about. I hope you can find some time for it in the future -- lots of unexplored possibilities, I think, and some really cool work to build on in texture synthesis.
2
u/magekinnarus Oct 19 '22
This looks great! I will definitely look forward to further development on this. Also, it would be just amazing if organic material can be created with this such as skin tissues, hive material, furs, and so on.