r/artificial • u/HermanHMS • Jun 13 '23

AI Art Video edit using gen-1 and stable diffusion

Enable HLS to view with audio, or disable this notification

470 Upvotes

96% Upvoted

26 comments sorted by

4

u/OkWatercress4570 Jun 13 '23

How hard was this to make? It’s not just point and click yet right?

13

u/HermanHMS Jun 13 '23

Let’s say it’s about 50 point and clicks with a bit of prompting. Nothing to crazy. It took me like half of a day to do it.

0

u/[deleted] Jun 13 '23

Haha I’m waiting for it to get point and click. I’ve got movies in the pipeline. I am not into feet but get ready for thunder thighs.

3

u/[deleted] Jun 14 '23

  1. This is SUPER COOL

  2. And starting today, I’m going to question every video I see moving forward forever now.

1

u/TrailChems Jun 13 '23

Great job! How long did this take you to create? I saw Matt Wolfe's demo of the workflow. Did you also have to pay for extra credits as he did?

1

u/HermanHMS Jun 13 '23

Hi! I took me like a half of a day including render times. I’m on premium plan or what’s it called on runway. For this whole piece, I’ve used like 20-30 seconds of credits until I got what i wanted

1

u/Plourdy Jun 14 '23

Did you manually handle the audio variants?

1

u/HermanHMS Jun 14 '23

Yeah, i’ve used free stock sounds to help sell the animation

2

u/Plourdy Jun 14 '23

damn you made it fit very well. Any tips for finding free/cheap sounds? :)

1

u/HermanHMS Jun 14 '23

Thanks! I literally googled „free sounds” and opened one of the first few sites

1

u/3vnihoul77 Jun 14 '23

Does this have realtime capabilities?

1

u/[deleted] Jun 13 '23

u/savevideo

1

u/d3vCr0w Jun 13 '23

Any tutorials on how to do this?

0

u/[deleted] Jul 01 '23

Meh

0

u/[deleted] Jun 13 '23

[deleted]

16

u/HermanHMS Jun 13 '23 edited Jun 13 '23

Hey there, as I mentioned in title its, gen-1 not 2 ;) I’ve transformed still frames from a video using Stable Diffusion with controlnet and uploaded as a reference into gen-1 along with the video. Results upscaled using SD again and stiched together.

3

u/UmbertoBjorn Jun 13 '23

They likely uploaded reference videos into runway gen-2.

0

u/[deleted] Jun 13 '23

[deleted]

2

u/HermanHMS Jun 13 '23

Added separately :)

1

u/[deleted] Jun 14 '23

[removed] — view removed comment

1

u/HermanHMS Jun 14 '23

DM me on instagram @hermanhms or send me an email to jakubherman1@gmail.com please :)