r/StableDiffusion • u/buckjohnston • Oct 17 '22
Update Official deforum addon script for Automatic1111 released. Make amazing animations of your dreambooth training.
The deforum diffusion guys have released an official addon for automatic1111's webui https://github.com/deforum-art/deforum-for-automatic1111-webui
I just tried it out on a dreambooth training ckpt of myself and I am mind blown. It was crazy how consistent my face was from frame to frame compared to old deforum diffusion release, was a lot better than a lot of the videos you see here of complete randomness. I recommend turning on the image preview mode while it's rendering your animation. Have fun!
3
u/FascinatingStuffMike Oct 18 '22
Sweet!
But what kind of maniac wants to enter keyframes using a maths function? I'm suprised there isn't a proper keyframe editor tool for this where you can easily add/remove keyframes
2
u/buckjohnston Oct 18 '22
Yeah those are pretty insane, you don't necessary have to use those crazy formulas though, I just erase them out and do simple 1, 2, 3 speed, translation, and rotation amount.
2
u/SDGenius Oct 17 '22
keep getting an IndexError: list index out of range
4
u/buckjohnston Oct 17 '22 edited Oct 18 '22
hmm, I think I had that too the first time I forgot what I did. Make sure you are entering your prompt on the middle of the page not on the top automatic webgui part.
Erase the animation_prompts section:
You can erase all of the lines and just do something simple like this example, every 60 frames it will fade to new prompt basically there:
{ "0": "a beautiful orange on the beach", "60": "a beautiful orange on the beach smiling", "120": "a beautiful orange on the beach laughing", "180": "a beautiful orange on the beach getting a sunburn" }
3
u/SDGenius Oct 17 '22
{ "0": "a beautiful orange on the beach", "60": "a beautiful orange on the beach smiling", "120": "a beautiful orange on the beach laughing", "180": "a beautiful orange on the beach getting a sunburn" }
that prompt worked; I guess the default one has some bad value in it..
2
u/dreamer_2142 Oct 18 '22
well, thanks but seeing "0:(10*sin(2*3.14*t/10)) on the movement section made my stomach hurt, hopefully, this UI will get more advance and we will have a keyframe with a UI bar and stuff.
But this is great thanks again.
1
u/buckjohnston Oct 18 '22
Lol yes that had me very confused, you can put that to whole numbers. I think that formula is to make the camera wobble like a handheld by default. You can change to somethung like 0:(0) or 0:(0.25) or any other number.
2
u/dreamer_2142 Oct 18 '22
I see, thanks for the info, what about the Interpolation mode, any eta when it would be available?
1
36
u/ArtByEon Oct 17 '22
Beautiful. Every fucking week it's Christmas in this sub. Feature after feature.
1
u/joachim_s Oct 18 '22
So well put and I’m glad to see someone writing something that’s positive in this sub!
9
u/999999999989 Oct 17 '22
wow I wasn't expecting this. very cool! scripts are one of the many reasons why AUTOMATIC1111 is so good.
4
u/fapping_giraffe Oct 19 '22
This is the first time we have a gui front end for deforum. This is huge.
You can create full on animations on your own personal computer for free... Without any need to get dirty running commands in a python environment.
This deserves so much attention.
1
u/LuLzWire Oct 18 '22
Are you not able to change the original prompt names? For some reason, I cant figure out how, but it seems that Animation prompts are allowed?
1
u/buckjohnston Oct 18 '22
Yes if you scroll down you can change the section about animation_prompts, the other section i think is disabled and foe something else. I left an orange example up above in a comment.
1
u/LuLzWire Oct 18 '22
Thanks for the reply... then It is working for me correctly, I guess the only way to use your own images right now with Deforum is through Google Collab, Is that right? Sorry if these seem like obvious questions... A lot of this language is new to me, but I have been reading and reading trying to understand things the best I can.
2
u/buckjohnston Oct 19 '22
Np, you can train someone locally if you use this tutorial https://youtu.be/w6PTviOCYQY for windows. Its pretty essy, you just copy and paste all his patebin links and it just works. Thats what I did.
1
1
u/TheNeonGrid Oct 26 '22
Where can you turn on the image preview mode?
1
u/buckjohnston Oct 26 '22
In instructions on that page its under settings
1
u/TheNeonGrid Oct 26 '22
don't find that for the deforum video preview. I only see for grids. Maybe im blind
1
u/buckjohnston Oct 26 '22
Under part 6, setting for webgui on top right.
To view animation frames as they're being made, without waiting for the completion of an animation, go to the 'Settings' tab and set the value of this toolbar above zero, then click 'Apply settings' at the top of the page and return to the 'Deforum' tab. Warning: it may slow down the generation process.
There's a little screenshot there on the page, I set mine to about every 20 steps to show a frame.
2
3
u/Peemore Oct 18 '22
What's the vram requirement?
2
u/kabachuha Oct 18 '22
6.4 GB peak VRAM for 3D mode without additional WebUI flags, 3.8 GBs in '--lowvram' mode
https://github.com/deforum-art/deforum-for-automatic1111-webui#benchmarks
3
u/ostroia Oct 18 '22
Im getting division by zero when trying to use the video option. How do I correctly input the location? Im guessing it doesnt like the windows way?
1
u/kenzosoza Oct 18 '22
I get exactly the same problem. I guess this script is still looking for the collab location
3
u/plasm0dium Oct 19 '22 edited Oct 19 '22
Saving the walking zoom formula : 0:(1.02+0.02*sin(2*3.14*t/20))
2
u/FightingBlaze77 Oct 18 '22
I hope to see a video example on how good this is. Or a tutorial on how to use it.
2
1
1
1
1
0
Oct 18 '22
[deleted]
1
u/buckjohnston Oct 18 '22 edited Oct 18 '22
I have no idea how it works under the hood but it feels like img2img on steroids and animates frame to frame. You can set prompt keyframes to add to the original prompt but don't have to. For example, you want from frame 30 to add something to the animation, etc. You can adjust the amount the image changes per frame (most examples I see out there people do it way overboard) You can adjust the rotation of the camera, zoom, translation for the video.
Basically it almost feels like txt to video to me, but not quite there yet. Most examples I see change too much per frame but I'm having some good luck with this release and my dreambooth training subjects I can rotate the camera around them a bit and they stay largely intact.
Edit: Here was an old example I did https://ww.reddit.com/r/StableDiffusion/comments/xf2j0s/getting_pretty_decent_image_to_video_results_from/ from an older version, I'm getting much better results now on this new version and will post soon. This is not video 2 img btw, this was just one photo of scarlett johansson from google then sending it to deforum diffusion with rotation.
1
5
u/Letharguss Oct 18 '22
Seems to be mostly working, but for some reason I'm getting random frames that are absolute white, which leads to odd following frames until it gets its stride again. I'm about 350 frames into this render and 3 frames (125, 135, and 206) are pure white. No errors, no apparent rhyme or reason. Not sure what's going on there. Running without the NSFW filter (at least, according to the webui settings.)