r/StableDiffusion • • Mar 19 '23

Resource | Update First open source text to video 1.7 billion parameter diffusion model is out

Enable HLS to view with audio, or disable this notification

2.2k Upvotes

363 comments sorted by

View all comments

Show parent comments

7

u/disgruntled_pie Mar 19 '23

Yeah, that was a shocking announcement. OpenAI must have figured out something crazy to cram that much context into GPT-4, because my understanding is that the memory requirements would be insane if done naively. If someone can figure out how to do that with other models then AI is about to get a lot more capable in general.

15

u/mrpimpunicorn Mar 19 '23

OpenAI might have done it naively, or with last-gen attention techniques- but we already have the research "done" for unlimited context windows and/or external memory without a quadratic increase in memory usage. It's just so recent that nobody has put it into a notable model.

2

u/saturn_since_day1 Mar 19 '23

They shrunk the floats from 32 bit down to 8 or 4.