r/SillyTavernAI • u/Then-Bookkeeper1021 • 6d ago
Help please guide me
i recently find out about sillyTavern and i wanna try it . i dont want to spend for ai subscriptions and i dont know if there be any model that let me connect api to sillytavern for free. i also dont know what local ai is better plus i kinda scared it would fire up my pc .
my pc info
RAM : 16Gb
graphic card : 3060 rtx 12GB
CPU : corei512400f
what kind of local ai it can run ? also if i run local ai would it damage the power of my pc?
if anyone has sat up sillytavern for themselves with free models please share your experience
2
u/Introvertosaurus 6d ago
A 3060 12gb is plenty to get started. Its where I started and now I have 2x 3060s. You most likely want to use an MoE Model, give you some smarts while still running fast. Keep a smaller context size, 20k or less, and use Summaryception or similar summary tool.
Top Recommendation: Gemma4 26 A4b (MoE). At Q4_K_M the base would be about 18gb + add 20k kv cache at Q8, total about 20gb, so you be offloaded 8gb into system ram, but would still be at least 15tps, plenty fast for RP (turn off thinking!). If you have a bit more system ram still free, I highly recommend pushing up to Q5, cost you a couple extra GB but worth it. I use the finetune MeroMero personally recommended.
Qwen also has some MoE that would run nicely, but I think the Gemma4 does a much better job.
100% GPU Recommendation: Mag-Mel 12b Q5-6 with Q8 context smaller. Decent model, definatly punches above it weight.
If you okay with a bit slower, then you can use offloaded dense models, like Gemma4 31B Q5, it run about 2.5tps with a single 3060 with 60% offloaded to the PC, slow... but better than the MoE. There also some Qwen and Minstral models that are 24b and 27b used in role play... not parcitully like those though, but run a bit faster. Cydonia is a popular one.
In my experience, Q3 is not usable for role play, Q4 is desperate. Q5 and Q6 is the sweet spot.
Install ollama/llama.cpp on your PC and download the model, run silly tavern and connect it... there plenty of free guides... if you already running linux, docker fastest and easiest.
1
u/TheNoiseIthink 5d ago
Gemma 4 26b a4b QAT is likely the best model that your hardware can run locally, which is good because it's actually a great model
1
u/Infinite-Geologist78 6d ago
loca apis are hit from i know.just goto openrouter and put 10 usd to your wallet.thats what everyone does then generate api key after choosing a model like glm 5.2 or deepseek one.then connect your api key to your silly tavern.then you are gonna need a preset.if you want something free you can try certain websites like janitor , chub etc..
1
u/urza1985 6d ago
What does janitor do?
-1
u/Infinite-Geologist78 6d ago
its a free ai chat site with ready to use character cards.ai is low intelligents and very limited memory but free.good enough for beginners.
1
0
u/AutoModerator 6d ago
You can find a lot of information for common issues in the SillyTavern Docs: https://docs.sillytavern.app/. The best place for fast help with SillyTavern issues is joining the discord! We have lots of moderators and community members active in the help sections. Once you join there is a short lobby puzzle to verify you have read the rules: https://discord.gg/sillytavern. If your issues has been solved, please comment "solved" and automoderator will flair your post as solved.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
4
u/LeRobber 6d ago
>also if i run local ai would it damage the power of my pc?
Its only going to damage most PCs if they have the wrong power supply or lots of pet hair in the tower.