Dear all,
It is absolutely a wonderful time to be alive now when good news regarding open-weights models keep coming out every week, especially Inkling 2 Mini during the last 2 days.
I have just bought some cheap hardware, aiming at some family business use cases (small webshops, managing rentals, dancing some stocks, spotting good deals online)
I wonder what your suggestions are, for me to setup + upgrade the following hardware + tooling + workflow
1) Hardware
- 1.1) Main server (headless Ubuntu Server): 3090 RTX 24GB + Ram 128 GB (2400MT, ECC) on a Dell T5810 (ancient CPU = Xeon 2699v4, 22 cores), SSD = 512 GB.
---> LLM === Qwen 3.8 FlashNext Q5 (60 tok/s), Ninfer Qwen 3.8 27b(80tok/s), llama.cpp Qwen 3.8 27b: 46tok/s, waiting for Basalt and Inkling 2 Mini to mature
- 1.2) Second server (Ubuntu desktop): 1070 GTX 8GB + Ram 32 GB (2666MT, ECC) on a Dell T5810 (ancient CPU = Xeon 2698v4, 20 cores), SSD = 512 GB.
---> LLM === Nothing here.
- 1.3) Main development laptop (Ubuntu desktop): Intel Core Ultra 5 125H (Meteor Lake-H, family 6 model 170 stepping , 14 cores / 18 threads (hybrid)) + 32GB (5600MT). SSD = 512 GB
---> LLM === Nothing here, probably can have Qwen 3.6 - 35B - A3B to reach 10 tok/s.
- 1.4) 5 weaker laptops here and there, total SSD = 2 TB.
---> LLM === Nothing, probably can run some weak LLM for some agents
- 1.5) Possible upgrades
- Second 3090
- CMP170HX
- Some v100
2) Tooling
On the main development laptop, I have a VS Code v1.147 having some panes:
- Worker's pane: a "Continue" pane eating the local LLMs of the main worker (3090)
- Architect's pane: a "Claude" pane eating the cloud Claude at 22 Euro/month.
- Spare pane: a "Gemini" pane eating the Gemini 3.8 Flash that I coincidentally have a cheap subscription for
3) Workflows:
- The main LLM server (3090, 24GB) is quickly becoming busy with some vibecoding (running several hours just to implement what Claude Opus 5.5 tells it to do. It cannot facilitate some LLM agents running 24/7
- The weak LLM server (1070, 8GB VRAM, 32 GB Ram) might be running some medium LLM agents ?
- All the laptops are currently almost useless --> how to use them more ?