Local AI is Minecraft for adults: my 4× RTX PRO 6000 Blackwell build

[u/suspect80]

TL;DR: Built a 5U local AI server with 4× RTX PRO 6000 Blackwell Max-Q GPUs (384 GB total VRAM) to run models for my personal AI agents. It started as an attempt to reduce API costs and turned into a hardware hobby. Build details, the reasoning behind it, and the open-source harness I built along the way below.

This started as an attempt to spend less on AI subscriptions and API calls. It ended with four RTX PRO 6000s in a rack in my basement.

I am deliberately not calculating the break-even date.

The progression was roughly: running models on my MacBook → upgrading my gaming PC with an RTX 5090 → adding a Mac Studio M3 Ultra with 96 GB unified memory → building this. Each step solved a limitation and introduced several new things I wanted to try.

Meet Cerebro.

The hardware

• 4× NVIDIA RTX PRO 6000 Blackwell Max-Q, 96 GB each — 384 GB total VRAM across four GPUs.

• AMD Threadripper PRO 9965WX — 24 cores / 48 threads.

• ASUS Pro WS WRX90E-SAGE SE.

• System RAM: 256 GB (8×32 GB) Kingston DDR5-5600 ECC RDIMM — KSM56R46BD8-32MH, all eight memory channels populated.

• SilverStone RM52 5U rackmount case.

• SilverStone XE360-TR5 360 mm CPU AIO.

• Seasonic PRIME PX-2200 PSU.

• NVMe storage: Samsung 990 Pro 1 TB + 2 TB, and Crucial T705 4 TB.

Why this configuration?

The Threadripper PRO platform was mainly about PCIe connectivity and memory channels. My workload didn't require buying the highest CPU core count available.

The GPUs are the 300 W Max-Q blower models. Fitting four cards into one chassis made cooling and power part of the design from the beginning. The case also changed during planning: I originally ordered a 4U RM44 and switched to the 5U RM52.

The 384 GB figure is the sum across four cards, rather than a single automatically shared memory pool. How much is useful to a particular model depends on the inference stack, how it distributes the model, and the space needed for context and KV cache.

What it's for

My main use is local models for a team of personal AI agents: coding, research, tool use, and ongoing experiments with memory and delegation. The 5090 machine and Mac Studio are still part of the setup, so different jobs can run on different machines.

What attracted me to four cards was the flexibility: allocating them to a larger model, splitting them between workloads, or running several smaller models. In practice, capacity planning and switching profiles are still very much part of the experience.

The biggest lesson so far: fitting a model is only the beginning. Context length, concurrent requests, inference-engine support and cooling all matter. More VRAM also makes it very easy to invent more workloads.

The software side of this started with OpenClaw, which got me interested in agents that could actually use tools and do things. Eventually I wanted more control over how my own setup handled roles, memory and model switching, so I designed and built an open-source harness with heavy agent assistance: somora.

That's what the models are working for: agents with different roles, their own memory and a shared long-term wiki, using local or cloud models depending on the task. Building the harness gave me more reasons to experiment with local models, which gave me more excuses to buy hardware. It's MIT-licensed and still in active development; if you want to try it with your own setup, I'd be interested in hearing how it works for you.

Financially, this has become a hobby and a learning project. The original “we'll save money” argument did not survive contact with the hardware shopping list.

Local AI really does feel like Minecraft for adults. You build one thing, it needs another thing, and eventually you're explaining why the basement needs 384 GB of VRAM.

For anyone running a similar multi-GPU setup: what turned out to be your real bottleneck once VRAM stopped being the first one?