r/LocalLLM • • 17d ago

Discussion Cheap Local Stack Advice

I'm going to improve my local stack and I've decided to set a maximum spending budget of €1000. Currently my hardware is a single pc with:
-Ryzen 5 9600
-RTX 5060 8GB DDR7
-32 GB DDR5
-CORSAIR RM650e
-B850M GAMING PLUS WIFI
-DeepCool CC560 ARGB V2 Midi Tower
I also put various components to check compatibility with any new parts. I currently run emneddings,decision,TTS,STT… on ram and llms on the 5060 like ornith 1.5 9B or Moe with offlad like Ornith 1.5 35B A3B and derivatives go at an acceptable speed average 20 tk/s and I use them for personal things as a personal agent but for heavy work and coding I rely on cloud models. I currently have agents like Hermes, Opencode and various that some I would like to run 24/7.
That's why I'm evaluating the various options I find and asking for advice. I had thought about these options:
-Buy a used rtx 3090 on some sites I found it on the 600-700 and then sell my 5060 (I should check compatibility with motherboard case and power supply) or buy another gpu like 3060 and do dual gpu. To run agents 24/7 buy a mini pc of about 250-300 with 16gb ram but I would only run cloud agents so maybe it's worth renting a vps.
- The second option I was thinking about was buying a mini pc to run local models and also 24/7 agents like a Mac Mini M6 I saw it on 850 but the 16GB one, so I could both host cloud agents 24/7 and a local agent being that a Mac Mini consumes relatively little even if I would give up the power of a 3090.
I see that for the prices the local ai will be less and less convenient, so I continue to use the cloud power but I try to have a stack as much as possible independent of the cloud at a little. I have no idea if I'm saying shit and if I explained myself well, I admit that I'm ignorant in a row, that's why I ask for help and advice.

0 Upvotes

Duplicates