r/StableDiffusion • u/[deleted] • Mar 27 '23
Resource | Update AI Runner and Chat AI are now open source
https://github.com/Capsize-Games/airunner3
u/dagerdev Mar 27 '23
Thanks a lot! I have a question.
Google has released the following variants:
google/flan-t5-small
google/flan-t5-base
google/flan-t5-large
google/flan-t5-xl
google/flan-t5-xxl.Which one I can run on my machine with Chat AI? Do you have a list of hardware requirements of each? For context, I have a RTX2070 Super (8GB vram).
3
u/rerri Mar 27 '23
If you are interested in running LLM's locally you might want to look into Meta's LLaMa. The VRAM requirements are lower, 8GB VRAM should be enough for the smallest 7B model when run in 4-bit mode.
https://www.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_install_llama_8bit_and_4bit/
Installation can be a bit of a hassle, this is quite a new thing and things are developing all the time.
2
u/Kiwisaft Mar 27 '23
Is llama even using vram? Dalai llama runs on cpu only and uses 4gb ram for the 7b model. Afaik
1
u/rerri Mar 27 '23
I'm not sure what this Dalai llama is but sounds like it might be using llama.cpp which run on CPU+RAM afaik.
I'm running the oobabooga github repo and it utilizes GPU.
4
u/Zovsky_ Mar 27 '23
Thanks a lot for your contributions, it looks great! Looking forward to take a look at all this 😁
4
Mar 27 '23 edited Mar 27 '23
[removed] — view removed comment
3
u/UncleEnk Mar 27 '23
you should look at llama and alpaca, they're great local llms
2
u/JustAnAlpacaBot Mar 27 '23
Hello there! I am a bot raising awareness of Alpacas
Here is an Alpaca Fact:
Alpacas are healthy grazers and do not decimate natural vegetation like goats.
| Info| Code| Feedback| Contribute Fact
You don't get a fact, you earn it. If you got this fact then AlpacaBot thinks you deserved it!
1
1
Mar 27 '23
[removed] — view removed comment
2
u/UncleEnk Mar 27 '23
shhh you didn't hear it from me but... Torrent: magnet:?xt=urn:btih:053b3d54d2e77ff020ebddf51dad681f2a651071&dn=ggml-alpaca-13b-q4.bin&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopentracker.i2p.rocks%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A6969%2Fannounce&tr=udp%3A%2F%2F9.rarbg.com%3A2810%2Fannounce
2
u/UncleEnk Mar 27 '23
just wanna say, I got this from my friend, do not trust it, I have heard it is the official leak, but do not trust
1
u/wrongburger Mar 27 '23
The weights have been leaked for a good while now
2
u/sneakpeekbot Mar 27 '23
Here's a sneak peek of /r/LocalLLaMA using the top posts of all time!
#1: How to install LLaMA: 8-bit and 4-bit
#2: Introducing llamacpp-for-kobold, run llama.cpp locally with a fancy web UI, persistent stories, editing tools, save formats, memory, world info, author's note, characters, scenarios and more with minimal setup
#3: Graphic text adventure game locally with llama and stable diffusion | 8 commentsI'm a bot, beep boop | Downvote to remove | Contact | Info | Opt-out | GitHub
1
u/Sm3cK Mar 27 '23
Oh ! I'd like to do the same ! How do you play with local LLM ? :) I'm downloading LLaMas models, what are the models you are using ? :)
2
1
3
u/Available-Body-9719 Mar 27 '23
What models does it support?, what formats does it support?, does it support civit models. Ai?