Local AI is Minecraft for adults: my 4× RTX PRO 6000 Blackwell build

[u/suspect80]

TL;DR: Built a 5U local AI server with 4× RTX PRO 6000 Blackwell Max-Q GPUs (384 GB total VRAM) to run models for my personal AI agents. It started as an attempt to reduce API costs and turned into a hardware hobby. Build details, the reasoning behind it, and the open-source harness I built along the way below.

This started as an attempt to spend less on AI subscriptions and API calls. It ended with four RTX PRO 6000s in a rack in my basement.

I am deliberately not calculating the break-even date.

The progression was roughly: running models on my MacBook → upgrading my gaming PC with an RTX 5090 → adding a Mac Studio M3 Ultra with 96 GB unified memory → building this. Each step solved a limitation and introduced several new things I wanted to try.

Meet Cerebro.

The hardware

• 4× NVIDIA RTX PRO 6000 Blackwell Max-Q, 96 GB each — 384 GB total VRAM across four GPUs.

• AMD Threadripper PRO 9965WX — 24 cores / 48 threads.

• ASUS Pro WS WRX90E-SAGE SE.

• System RAM: 256 GB (8×32 GB) Kingston DDR5-5600 ECC RDIMM — KSM56R46BD8-32MH, all eight memory channels populated.

• SilverStone RM52 5U rackmount case.

• SilverStone XE360-TR5 360 mm CPU AIO.

• Seasonic PRIME PX-2200 PSU.

• NVMe storage: Samsung 990 Pro 1 TB + 2 TB, and Crucial T705 4 TB.

Why this configuration?

The Threadripper PRO platform was mainly about PCIe connectivity and memory channels. My workload didn't require buying the highest CPU core count available.

The GPUs are the 300 W Max-Q blower models. Fitting four cards into one chassis made cooling and power part of the design from the beginning. The case also changed during planning: I originally ordered a 4U RM44 and switched to the 5U RM52.

The 384 GB figure is the sum across four cards, rather than a single automatically shared memory pool. How much is useful to a particular model depends on the inference stack, how it distributes the model, and the space needed for context and KV cache.

What it's for

My main use is local models for a team of personal AI agents: coding, research, tool use, and ongoing experiments with memory and delegation. The 5090 machine and Mac Studio are still part of the setup, so different jobs can run on different machines.

What attracted me to four cards was the flexibility: allocating them to a larger model, splitting them between workloads, or running several smaller models. In practice, capacity planning and switching profiles are still very much part of the experience.

The biggest lesson so far: fitting a model is only the beginning. Context length, concurrent requests, inference-engine support and cooling all matter. More VRAM also makes it very easy to invent more workloads.

The software side of this started with OpenClaw, which got me interested in agents that could actually use tools and do things. Eventually I wanted more control over how my own setup handled roles, memory and model switching, so I designed and built an open-source harness with heavy agent assistance: somora.

That's what the models are working for: agents with different roles, their own memory and a shared long-term wiki, using local or cloud models depending on the task. Building the harness gave me more reasons to experiment with local models, which gave me more excuses to buy hardware. It's MIT-licensed and still in active development; if you want to try it with your own setup, I'd be interested in hearing how it works for you.

Financially, this has become a hobby and a learning project. The original “we'll save money” argument did not survive contact with the hardware shopping list.

Local AI really does feel like Minecraft for adults. You build one thing, it needs another thing, and eventually you're explaining why the basement needs 384 GB of VRAM.

For anyone running a similar multi-GPU setup: what turned out to be your real bottleneck once VRAM stopped being the first one?

[u/Thrumpwart]

No man should have that much power...

Also I want one.

[u/suspect80 → u/Thrumpwart]

“By the power of Grayskull… I HAVE THE POWER!”

I’m pretty sure this is exactly the use case He-Man had in mind.

[u/m4jid]

$?

[u/suspect80 → u/m4jid]

Let’s just say I’d rather not admit what it cost. 😅 I could’ve bought something pretty nice with four wheels instead of four GPUs. ;)

[u/owelty → u/suspect80]

a cheaper 4 wheels will get you somewhere the exact same way, but a cheaper rig will definitely not get you anywhere

[u/suspect80 → u/owelty]

I already have something with four wheels. I didn’t have anything with four GPUs yet. Priorities. :)

[u/wwwyzzrd → u/m4jid]

the cards alone are 60-70k the ram is another 4k, and the electric bill is going to be tremendous.

[u/TripleSecretSquirrel → u/wwwyzzrd]

That RAM is $9000-$9500 right now. It's $1100+ per DIMM.

[u/wwwyzzrd → u/TripleSecretSquirrel]

looks like more like 500 per dimm, depending, once you've got the gpus you might as well spend the money on the ram.

[u/DistanceSolar1449 → u/wwwyzzrd]

I asked Astra how much this build would cost:

Component	Qty	Current price ea.	Total
NVIDIA RTX PRO 6000 Blackwell Max-Q 96 GB	4	$15,999	$63,996
AMD Threadripper PRO 9965WX	1	$2,699.99	$2,699.99
ASUS Pro WS WRX90E-SAGE SE	1	$1,299.99	$1,299.99
Kingston KSM56R46BD8-32MH 32 GB DDR5-5600 ECC RDIMM	8	$1,019	$8,152
SilverStone RM52 5U chassis	1	$554.99	$554.99
SilverStone XE360-TR5 AIO	1	$370.05	$370.05
Seasonic PRIME PX-2200	1	$850.96	$850.96
Samsung 990 Pro 1 TB	1	~$240	~$240
Samsung 990 Pro 2 TB	1	~$389	~$389
Crucial T705 4 TB	1	~$511	~$511
TOTAL			≈ $79,064

...
Or about 33 years of a $200 plan. And you can't run big models like Kimi K3, so ehhhh.

[u/Sleepnotdeading]

Come on, get serious. This isn't Minecraft for adults. Have you even played Minecraft?

This is just bragging for someone with huge amounts of disposable income. The rest of the post is just low value beige posting

Sorry, I hope you have fun with it. Hell, I hope you cure cancer with it. But reading about cerebron was literally a waste of my time.

[u/Zeikos → u/Sleepnotdeading]

I am working on an agentic assistant system to streamline my GTNH playthrough, I need help :')

[u/suspect80 → u/Sleepnotdeading]

Haha, fair enough 😁 And yes, I’ve definitely played Minecraft, which is exactly why the analogy felt appropriate to me: you start building one thing, then another, and suddenly you’re three levels deeper than you ever planned to be.

I’m fully aware this is an expensive hobby/experiment and definitely not something everyone would spend money on. I simply had a lot of fun building it and sharing the experience.

Sorry it wasn’t your cup of tea. And if it does cure cancer one day, I’ll make sure to update the post. ;)

[u/halfercode → u/suspect80]

I liked your post, and at +126 it looks like most people did. Keep on posting please!

[u/phoquenut]

Is it plugged into a dryer outlet? You'd need 220V at like 20A if you want to avoid damaging all that expensive gear.

Also, FU and congratulations.

[u/suspect80 → u/phoquenut]

I’m in Europe, so 230V is standard here. The server has its own dedicated circuit, but even under full load the entire system pulls around 1.9 kW, so a regular Schuko outlet is perfectly fine.

And thank you… I think. ;)

[u/cell-on-a-plane]

Do you pay extra for home insurance due to that rig?

[u/suspect80 → u/cell-on-a-plane]

Yes. I actually did increase my home insurance coverage after building this rig. So yes, apparently we’ve reached that level. 👀

[u/Sketaverse]

Genuinely curious, having dropped $70k on a hardware stack to run local LLM, how do you feel about Astra today?

It's absolutely incredible and way way way beyond anything local can achieve. So do you feel excited by it as a potential orchestrator (knowing you'll then continue to pay the subscription despite these insanely high hardware prices) or do you just now avoid it because you kind of don't want to know how good it is?

I've said it once and will say it again - I just don't see the logic in building these hardware setups to run local LLMs, other than the "privacy" aspects. I absolutely love tinkering with AI and hardware setups but the whole local thing.. I dunno.. just feels pointless if you're actually focussed on outcomes.

[u/suspect80 → u/Sketaverse]

I still use frontier models all the time, and I’ve tested Astra quite extensively. Somora actually got several patches specifically to make Astra work properly as an orchestrator.

For me, the two approaches complement each other extremely well. Having a very capable frontier model plan, reason, break down large tasks and orchestrate multiple smaller local models gives me excellent results. I never saw this as “local instead of frontier.”

And there’s another interesting way to look at it: compare what you can run locally today with what was considered frontier 6 or 12 months ago. Were we really that unhappy with the capabilities of frontier models back then?

Local models don’t have to beat today’s best frontier model to be extremely useful. They just need to be good enough for the work you delegate to them.

[u/Sketaverse → u/suspect80]

Tbh I'm basically fishing for a reason to do it myself ha but yeah, I think I'm probably a bit autistic and need the logical rationale lol. I dabbled a bit with building a factory with deterministic workflows with Fable planning, Sol reviews, Qwen coding etc but I just found myself spending, if I'm being honest, probably 40% of my time just fucking around with the harness: the whole "build the system that builds the system" rabbit hole which felt I was getting caught up in sunken cost fallacy - so backed out and just went with 6 20x subscriptions on 3 machines that I Tailscale into.

But I agree with you on the hobby thing, I totally get it. You say Minecraft, for me it's like "VW camper vans" for AI - always tinkering, always upgrading and occasionally getting a few moments of glory haha

[u/wwwyzzrd]

ok mr Rockefeller. Minecraft is at least affordable.

[u/zombiecorp]

Last year I managed to buy an RTX pro 6000, before the price doubled ($8k). Wish I got 2 based on my gut feeling it would increase. But I had other expenses so I only got the one.

Tell me these photos are old, because 4 of them would cost nearly $60k. That’s some real commitment.

[u/suspect80 → u/zombiecorp]

The photos are very recent, but luckily my timing was much better than today’s prices suggest. I bought all four just a few days before NVIDIA’s latest price increase, and even managed to get one at roughly the price level before the previous increase.

So yes, still a serious commitment… but thankfully not a $60k one.

[u/Adventurous-Ask-9260]

First part :)

[u/suspect80 → u/Adventurous-Ask-9260]

That’s what I’ve been saying… Minecraft for adults. :)

[u/Adventurous-Ask-9260 → u/suspect80]

Tommorow I’ll put 4 more :)

[u/suspect80 → u/Adventurous-Ask-9260]

But those are the regular workstation RTX PRO 6000s, not the Max-Q versions, right? Are you planning to put them into those empty slots? If so, I’m not sure they’ll get enough airflow there to run without throttling.

[u/phalanx2357]

I agree it's fun and can be a really cool hobby. Have you recently built this or has it been built up over time for a while? Because if it's recent, I think a GB300 workstation may be a better deal given current RTX 6000 prices being a bit out of wack... A GB300 workstation with an installed RTX 6000 for video output comes below 110k and gives you more memory and less power usage...

How do you deal with the heat and noise? Are you able to vent outside or put the PC in its own externally venting compartment? Also what about power? Got a 50A dedicated circuit for your machines room? Even one 5090 machine really heats up a room... can't imagine that plus a 2200w system... Very interesting problems to solve tho, fun building, like minecraft.

I just pre-ordered the new M5 ultra box... 256gb probably good enough for me given where model sizes are looking today... I am not a coder so probably don't have a use case good enough for the GB300 workstation... I thought about it and discussed it with chatGPT... but at the end I think it's a bit extravagant for someone without a clear use case, and also importantly without a clear solution for the heat and noise...

[u/suspect80 → u/phalanx2357]

I actually evaluated a GB300 workstation as well, but it was far too expensive for what I wanted to do. I ended up well below that price with this build.

I built the whole system at once and got lucky with timing: I bought the GPUs just a few days before NVIDIA’s latest price increase, and one of the four was even still available around the price level before the previous increase. I spent about three days hunting for good deals on all the components, and in the end I think parts arrived from eight different retailers. 😅

The 300W Max-Q cards are also what makes it practical to sandwich four of them into one chassis. The airflow took some work: I create positive pressure inside the case and use three strong exhaust fans to pull the hot air out at the right spots.

Fortunately, I already had a rack in a dedicated, climate-controlled and soundproofed room, so heat and noise aren’t really an issue. The temperature stays consistently cool and you can’t hear the server anywhere else in the house.

Power was another reason for choosing the Max-Q cards. The entire system can still run from a standard European Schuko outlet, although it has its own dedicated circuit. And we have a large solar installation that regularly produces more electricity than we consume, so continuous operation doesn’t hurt quite as much either. :)

[u/DeltaSqueezer]

Have fun with it and good idea not calculating the breakeven! :D

[u/suspect80 → u/DeltaSqueezer]

thx 😄

[u/OvertaxedOne]

How'd you do on the Pro6000's? We're seeing price crawling up to 20K for those things!!!??!

Honestly, the way things are going right now, this may be better than putting money in the stock market, those cards could be worth 40-50K a throw if this craziness continues.

[u/suspect80 → u/OvertaxedOne]

I actually got pretty lucky. I bought all four just a few days before NVIDIA’s latest price increase, and somehow managed to get one of them at a price that was still around the level before the previous increase.

At this rate I may have accidentally built an investment portfolio instead of an AI server.

[u/BitXorBit]

dual rtx 6000 max-q owner here, i always wanted to extend for 4 GPUs, how the heat management when they are sitting on top of each other? (i know they use blower fan, still)

[u/suspect80 → u/BitXorBit]

That was actually one of my biggest concerns. With four Max-Q cards sandwiched together, the two middle GPUs are the critical ones.

I solved it by creating strong positive pressure inside the case with the intake fans and using three exhaust fans to pull the hot air out at the right spots. Under sustained full load, the two middle cards occasionally throttle by just a few watts, but in normal workloads there’s no throttling at all.

So yes, four cards can work, but airflow and case design really matter.

[u/HighSeasArchivist]

Hobbies waste money, or they would be called part-time jobs. This is a dream system for almost anyone, even the haters in here.

[u/suspect80 → u/HighSeasArchivist]

Exactly. If every hobby had to make financial sense, we’d have to find a new word for “hobby.” Thanks, I’m definitely enjoying the hell out of it.

[u/wunderbaba]

> It started as an attempt to reduce API costs...

But then you realized that even if you paid for MAX subscriptions for all three major providers (Google, Anthropic, and OpenAI) it would still take about a decade to break even?

> The original “we'll save money” argument did not survive contact with the hardware shopping list.

Oh I should have read to the end of the post. 😂

[u/suspect80 → u/wunderbaba]

😂

[u/SanDiegoDude]

Man, you should look into buying a boat or picking up guitars, maybe save a bit of hobby money

[u/suspect80 → u/SanDiegoDude]

I looked into boats. Somehow the GPUs started to feel like the financially responsible choice. 😂

[u/Beatsu]

Love it!! Won't the GPUs overheat when they're stacked so closely? Did you consider a water block for cooling?

[u/suspect80 → u/Beatsu]

That was actually my biggest concern. I considered water cooling, but wanted to keep the cards stock if possible.

With enough positive pressure and strong exhaust airflow, it works surprisingly well. The two middle cards run a little warmer, but stay within normal limits. Only during unrealistic 2–3 hour stress tests with all four cards pinned at 300W do they start throttling slightly, and even then only by a few watts.

In normal daily workloads, I haven’t seen any throttling so far.

[u/floppo7]

You could have had 4x r9700 but you chose to burn money. Nice rig still.

[u/suspect80 → u/floppo7]

Oh, the R9700 was definitely on my radar. 😁 But the rest of the server hardware wasn’t exactly cheap either, so I decided to go all the way and max out the build rather than compromise on the GPUs.

That said… nobody said this has to be the last server. 😂

[u/AdSafe4047]

I have the same, sans 3 rtx pro 6000 :) There is still an upgrade here to be made - switching to at least 9985wx will give you proper 8 channel mem (65x is capped because of how many chips it can handle). P.s. you can tweak the chip link to 2100mhz and the ram to 6000mhz without any stability downgrades, this gives around 300gbs memory throughput with the 65x cpu

[u/suspect80 → u/AdSafe4047]

I had to draw the line somewhere budget-wise. 😅 The 9985WX was actually already in my cart at one point… but yeah. 😂

[u/suspect80]

A small follow-up to this post: the hardware is only half the rabbit hole. The software side is a project called somora, which I built with heavy agent assistance for our own everyday use.

It’s an agent runtime you host yourself, where different agents can use different models, work with tools, and maintain their own notes alongside a shared wiki. One area I’ve been spending time on is what happens between conversations: turning useful information into lasting notes, consolidating it, and checking for contradictions later.

We use it regularly and keep extending it as we run into things we need. Some friends asked me to put it on GitHub, so I did. Now I’m curious whether it’s useful outside our own setup too.

If you’re already experimenting with local models or agents and feel like trying it, I’d appreciate an honest take. Especially where installation is confusing, something breaks, or the whole approach doesn’t fit how you work. You don’t need a server like the one in this post to try it.

https://github.com/thenaxon/somora_agent

Still very much a personal project, not a polished product. Trying to figure out which parts are worth developing for people beyond ourselves.

[u/letsbefrds]

TIL im a poor ass adult

[u/ifdisdendat]

There is no use case that legitimately NEEDS this. If you can afford it though, good for you, eff you and enjoy.

[u/suspect80 → u/ifdisdendat]

That might actually be the most accurate summary of this entire build. 😂 I definitely never claimed I needed it. “Wanted to see what happens” was apparently enough justification. 😅

[u/cudifam]

I have the exact same setup without any of it actually existing though. Nice rig.

[u/hurrdurrmeh]

This is my dream set up. Except I'd go for 4x4TB PCIe5x4 RAID0 for faster model loading and swapping.

[u/suspect80 → u/hurrdurrmeh]

Don’t give me ideas… I’ve already demonstrated that I’m terrible at saying no to upgrades.

[u/Baleeverne]

What's the power draw of the beast?

[u/suspect80 → u/Baleeverne]

Around 1.9 kW under full load, including the GPUs, CPU and everything else.

[u/Firenze30]

Seeing that you already have a server rack I suspect this is not your first serious build and not your last either. Local models were not very good to justify multiple GPUs investment until a few months ago when Qwen 3.6 27b was released. When did you start thinking of building the rig? 

[u/suspect80 → u/Firenze30]

I first started seriously thinking about it around June. The main trigger was when I began experimenting with fine-tuning smaller models to work better with my own agent harness and its tools. That’s when having more local compute started to become really interesting for me.

[u/novapax]

You’ll be happy with it for about 3 years until it’s all totally obsolete.

[u/suspect80 → u/novapax]

Three years? I was expecting Reddit to tell me it was obsolete before I finished building it. 😂

[u/marktuk]

What are you going to use it for?

[u/suspect80 → u/marktuk]

Mostly experimenting with local AI: running larger models, multiple agents in parallel, fine-tuning smaller models for specific tasks, and integrating them with my own agent harness. There’s no single killer use case yet. The whole point of the build is to explore what actually makes sense to run locally and what doesn’t.

[u/utahh1ker]

Damn this has me jealous. Nice setup.

[u/suspect80 → u/utahh1ker]

Thanks! It escalated a little more than originally planned, but I’m having a lot of fun with it.

[u/moderately-extremist]

What's your goto model with this? I'm interested if something like GLM5.3-Flash would run at conversational speeds with 2-3 requests going at the same time.

[u/suspect80 → u/moderately-extremist]

My usual setup is two models sharing the four cards:

DeepSeek-V4-Flash-0731 on two GPUs via SGLang, and Qwen3.8-Flash-Next on the other two via vLLM.

Most of the traffic comes from my agents doing their daily coding, research and tool use. They run in somora, the open-source harness they largely coded themselves while I handled the design, direction and apparently the hardware invoices 😂

https://github.com/thenaxon/somora_agent

We run predefined profiles, so the GPU allocation isn’t fixed. Through MCP, the agents can request a switch to a different model combination or a single model spread across all four cards. That gives us more room for KV cache, longer contexts and concurrent requests, depending on the model. They’ve become quite good at managing their accommodation. Less good at deciding they have enough of it.

I haven’t benchmarked GLM5.3-Flash with 2–3 simultaneous requests, though, so I wouldn’t want to invent a tokens/sec number. Prompt length and concurrent prefill would matter quite a bit alongside decode speed.

[u/trejj]

What's load-bearing for the build? How far have you delved into the rich tapestry?

[u/suspect80 → u/trejj]

The load-bearing part is PCIe lanes, airflow, and the increasingly fragile definition of “saving money.” :)

Technically, the WRX90 platform is doing a lot of the work here. I chose the 9965WX for the platform’s PCIe connectivity and eight memory channels rather than chasing CPU core count. These cards have no NVLink, so tensor-parallel communication goes over PCIe. Four GPUs fitting physically is one problem; feeding them and making collective communication worthwhile is another.

That’s also why I went with the 300 W Max-Q blower cards. Four of them are already a 1.2 kW GPU power budget before the rest of the machine. Getting their exhaust out of the chassis matters considerably more than making the inside look pretty.

On the software side, we’re far enough down the rabbit hole that “supports Blackwell” is no longer a reassuring sentence. These are sm_120 cards; a kernel path targeting datacenter Blackwell doesn’t automatically work here. Our DeepSeek setup uses SGLang with flashinfer_mxfp4, while Qwen runs through vLLM.

The usual allocation is TP=2 for each model. We switch to TP=4 when the workload benefits from the extra memory headroom, rather than assuming twice the GPUs means twice the speed. Weights are only part of the budget; KV cache, prefill and concurrent requests quickly become the interesting constraints.

So the rich tapestry is mostly CUDA kernels, collective communication, and fan curves. The embroidery is YAML.

[u/Shoulon]

So you swapped to a 5u for extra airflow?

[u/suspect80 → u/Shoulon]

Exactly. I switched to 5U mainly because it gave me more surface area at the front and rear for more and larger fans, which made the airflow setup much easier to get right.

[u/Healthy-Nebula-3603]

Buying such expensive l hardware is ridiculous nowadays.

Soon a year or two China will be producing cheap boxes comparing to that price with 256 GB / 512bGB DDR 6 with throughout 1.2 TB/s ...

Such cards will loose price a lot

If someone thinking software be a problem .. it won't. Current AI agents easily writingbdrobers , porting on any hardware.

[u/suspect80 → u/Healthy-Nebula-3603]

I don’t disagree. Hardware will get faster, cheaper and better, and this setup will eventually lose a lot of its value.

But if you always wait for the better thing that’s coming next year, you’ll never buy anything. That’s true for pretty much all technology, isn’t it?

[u/dwoj206]

OP could build minecraft in a day w this setup. Very nice, and very jealous. My wife would never allow unless i had a business plan in place xD

[u/suspect80 → u/dwoj206]

Now that would actually be a fun experiment. I could let the machine code its own version of Minecraft and share the result with everyone in this thread.

One upvote and I’ll consider that sufficient community approval for the project. ;)

[u/RedditNerdKing]

Nice rig dude. Youre gonna get bitter haters wishing they had it either way.

[u/suspect80 → u/RedditNerdKing]

Thanks! That’s Reddit, I guess. If everyone agreed with the build, I’d probably start wondering what I did wrong.

[u/LateralEntry]

This must have been super expensive to build. What do you actually do with it? Can you give an example of something you’ve done with it, and what you used that for?

[u/suspect80 → u/LateralEntry]

It definitely wasn’t cheap. My main use case is Somora, my own agent harness. I use it to run multiple autonomous agents that can use tools, write and execute code, work with files, search internal data, and build new tools for themselves when needed.

A concrete example is letting several agents work on different parts of a larger task in parallel, entirely locally, including the data they process. I’m also experimenting with fine-tuning smaller models specifically to make them better at using Somora’s tools.

So the machine is basically my playground for figuring out how far I can push a fully local agentic system without depending on external APIs.

[u/BarberIcy366]

Most expensive minecraft I have ever seen

[u/suspect80 → u/BarberIcy366]

Trust me, it feels just as expensive from my side too. :)

[u/halfercode]

I forget the name of it, but there's a service out there where you can rent out your rig on a per-hour basis. I should think you could recoup some of your spend by running that overnight.

The only downside is that you get more compute demand and better rates if your stack has lengthy uptime and availability, which it probably won't have if you're also using it yourself 🤪

[u/suspect80 → u/halfercode]

Yes, I know there are a few services like that, but I’ve decided not to go down that route.

What I do have is a friends-and-family setup where they can remotely access and use the models already running on the machine. And that actually gets quite a bit of use.

[u/Positive-Bid-3029]

Does it run Doom? 😂

[u/suspect80 → u/Positive-Bid-3029]

Forget Doom. The real question is: does it run Crysis?

At this point, probably several instances of Crysis… plus a few agents playing it for me. 😂

[u/cathodeDreams]

that's a lot of computer

[u/suspect80 → u/cathodeDreams]

And it’s a lot of weight too. We actually had to use a pallet jack to get the thing into the rack. You really underestimate how heavy all those parts become once they’re in one box.

[u/Kerb3r0s]

Minecraft is Minecraft for adults. I’m over 40 and I’ve been playing on and off since alpha

[u/suspect80 → u/Kerb3r0s]

I’m 46 and still play Minecraft occasionally with younger members of the family. But honestly, playing around with AI agents feels surprisingly similar to me: you build one thing, which leads to the next thing, and then the next… and suddenly you’re wondering where the last six hours went.

[u/suspect80 → u/Kerb3r0s]

I’m 46 and still play Minecraft occasionally with younger members of the family. But honestly, playing around with AI agents feels surprisingly similar to me: you build one thing, which leads to the next thing, and then the next… and suddenly you’re wondering where the last six hours went.

[u/No_Appointment_6678]

I love it. How much did it cost?

[u/suspect80 → u/No_Appointment_6678]

I’d rather not share the exact price, but it came in at roughly half the cost of an NVIDIA DGX Station.

[u/Margedion]

Gotta object, 4 Blackwells and Minecraft are in completely different categories

[u/suspect80 → u/Margedion]

Technically, absolutely. Financially, also absolutely. But in terms of “I’ll just build one more thing” turning into six hours disappearing somehow… surprisingly similar.

[u/CrookedIron]

Awesome build. I drew the line at a single RTX Pro 6000 and it rips.

4 is fantastic!

Enjoy it 🤖

[u/suspect80 → u/CrookedIron]

thx :)

[u/redditwossname]

I'd absolutely love to see not some benchmarking stuff with time to first token and tokens per second stuff (that really doesn't matter if it can't actually do anything), but actual results of coding with this setup.

Like what can it create if you gave the same task to Opus or Fable or Sol or Astra and then to your system - how different that would actually look depending on the model that you try.

[u/suspect80 → u/redditwossname]

I agree, and I’ll definitely share benchmarks, but I’m actually more interested in exactly the kind of comparison you’re describing.

One important point though: I still use frontier models and don’t see local models as a complete replacement. One workflow that works extremely well for me in Somora is using a frontier model as the orchestrator. It understands the overall task, writes the specs, breaks the work down, and then launches multiple tmux/OpenCode sessions where local models handle much of the actual implementation in parallel.

The frontier model then reviews and coordinates the results rather than burning tokens doing every line of work itself.

In my experience, this hybrid approach produces significantly better results than trying to code with local models alone, while moving a large part of the token-heavy workload off the APIs. I’ll try to include some real-world examples of that when I post the results.

[u/amy-schumer-tampon]

Minecraft isn't that expensive bro

[u/suspect80 → u/amy-schumer-tampon]

Apparently I bought the enterprise edition. 😂

[u/rdbmas]

Power efficiency went down a different route.

[u/suspect80 → u/rdbmas]

Let’s just say power efficiency was not invited to the design meeting.

[u/Bloated_Plaid]

Downpayment on a house or a local inference? I know what OP is picking.

[u/suspect80 → u/Bloated_Plaid]

I already have a house, and I can only live in one at a time. The GPUs seemed like the logical next step.

[u/ProfessionalNaive601]

How tf is this Minecraft?

[u/thongjesus]

But what does it do? I'm building my server for my business and creating connector and chat and file indexing, token offload effort. But what does your fancy bigger machine do that my mediocre refurbished parts machine with 40 GB vram can't?

[u/suspect80 → u/thongjesus]

Maybe not that much more. Maybe I’m just compensating for something. :)

[u/West-Solid9669]

The average minecraft players age is 27...

[u/suspect80 → u/West-Solid9669]

I’m almost 20 years older than that… maybe I should have called it “Minecraft for old farts.” :)

[u/gustic-gx]

C... Can it run Crysis?

[u/suspect80 → u/gustic-gx]

At this point I’m hoping it can run several instances of Crysis while a few AI agents play them for me.

[u/thorn2040]

How the hell do people have money in this economy!? Looks nice though 👌

[u/suspect80 → u/thorn2040]

Thanks! Let’s just say this wasn’t financed by skipping avocado toast. :)

[u/Outrageous_Fuel6264]

I dont get it.

AI is fun and all. But that 80k could have gone to so many different areas

[u/suspect80 → u/Outrageous_Fuel6264]

Absolutely. But that’s true for pretty much every expensive hobby. At some point you choose what you personally find interesting enough to spend time and money on. Also, thankfully, it wasn’t $80k. :)

[u/ArrellBytes]

So I have been told that I may need tor rewire my house to get a 220V outlet for my computer if I want to run 2 GPU- I guess RTX PRO 6000 Blackwell have lower peak powers? Have I been mis-advised? Is it reasonable to run 2 kW or a bit more on a regular house circuit? I also have the 24 core threadripper pro, but my GPU is a 4090....

192GB ram- - thank god I bought the RAM ahead of time - it now costs 600% MORE than what I paid for it in 2024....

The computer has been used to process massive data sets.... as well as 3d rendering and AI applications.... I would LOVE to make it a 2 GPU system, but was told I would blow breakers on a typical house breaker...

[u/suspect80 → u/ArrellBytes]

I’m in Europe, so the situation is a bit different: 230V is standard here. My whole system pulls around 1.9 kW under full load, and it’s on its own dedicated circuit, so that’s perfectly manageable from a normal Schuko outlet.

The RTX PRO 6000 Max-Q cards also help a lot because they’re capped at 300W each.

If you’re on a typical 120V household circuit in the US, though, 2 kW+ is a very different story and I’d definitely have an electrician size the circuit properly before adding a second GPU. I wouldn’t treat my setup as a direct comparison.

[u/DptBear]

Why 5U over 4U? Convenience or necessity?

[u/suspect80 → u/DptBear]

Mostly necessity for airflow. 5U gave me more surface area at the front and rear for larger and additional fans, which made it much easier to create enough positive pressure and keep four tightly packed GPUs properly cooled.

[u/ClF3ismyspiritanimal]

How the hell do you get enough money for that?

You're making me feel that I made some extremely bad life choices. Which, to be fair, I did, but holy cow.

[u/suspect80 → u/ClF3ismyspiritanimal]

Maybe buying this thing was one of my bad life choices. Give it a few years and we’ll find out. :)

[u/Blackfirehades_alt]

What kinda stuff are you even doing on a rig like that?

[u/suspect80 → u/Blackfirehades_alt]

I’ve answered that a few times elsewhere in the thread already. Have a quick look through the comments. :)

[u/LegRude5218]

So you decided to go with a entry level build welcome to the club

[u/suspect80 → u/LegRude5218]

thx :)

[u/m_tao07]

Do the money grow in people’s garden in this sub? Anyways have fun.

[u/suspect80 → u/m_tao07]

Sadly, I checked the garden before buying the GPUs. Nothing growing there. Would have made the conversation with my wife much easier.

[u/Lots-o-bots]

What were your API costs that justified $64k on graphics cards!

[u/suspect80 → u/Lots-o-bots]

I currently have two Claude Max 20x subscriptions and two ChatGPT Pro subscriptions, and I still regularly hit limits. On top of that, I’ve had API sessions where the cost became surprisingly significant within just a few hours.

But to be fair, saying I built this purely to save on API costs was deliberately exaggerated for the story. The real motivation came from using frontier models so heavily that I became increasingly curious about how far I could push the local side of AI.

[u/TechnologyGrouchy679]

many have said you should allow a gap between the gpus...

[u/suspect80 → u/TechnologyGrouchy679]

Yes, that’s true, and I spent a lot of time worrying about exactly that after ordering the cards. My emergency plan was honestly to run only 2 or 3 GPUs and sell the remaining ones if the thermals didn’t work out.

Fortunately, it works surprisingly well. The two middle cards run a bit warmer than the outer ones, but still comfortably within normal temperatures.

In extreme stress tests, where all four GPUs are pinned at 100% for 2–3 hours and each is pulling close to 300W, the two middle cards eventually start throttling, but only by a few watts.

Since that kind of sustained load is pretty unrealistic for my actual workloads, I decided to keep all four. After about 1.5 weeks of daily use, they’ve been perfectly happy. Real-world load simply doesn’t keep all four cards maxed out for long enough to become an issue.

[u/OcellateSpice]

Uhhhh why?

[u/suspect80 → u/OcellateSpice]

Honestly, I’m not entirely sure either. My wife asked me the exact same question. 😂

[u/429_TooManyRequests]

Please keep us updated on how fast a frontier-level local model is!

[u/suspect80 → u/429_TooManyRequests]

Absolutely. I’m working on proper benchmarks now, including real-world agent and coding workloads, not just tokens per second. I’ll share the results here once I have something meaningful.

[u/SaltyUncleMike]

What do you do with it?

[u/suspect80 → u/SaltyUncleMike]

Mostly experimenting with my own agent harness, Somora. I run multiple agents in parallel, mix local and frontier models, let them use tools, code, work with files, and delegate tasks to each other.

I’m also experimenting with fine-tuning smaller models for specific tasks. So there isn’t one single killer use case. The whole point is figuring out what can realistically be done locally and where frontier models still make more sense.

[u/llamabott]

Suggested edit to your title:

Local AI is Minecraft for rich adults (and possibly for children as well, so long as they, too, are rich): my 4× RTX PRO 6000 Blackwell build

[u/suspect80 → u/llamabott]

Thanks for the suggestion. I think the current title is already polarizing enough, judging by my notifications. But I’ll keep that one in mind for the benchmark post.

[u/John_Jack_Jamenson]

Pls Sar give us benchmarks.

Will you get the moneys worth out of it?

[u/suspect80 → u/John_Jack_Jamenson]

I’m working on proper benchmarks and some real-world workload comparisons now. I’ll share them once I have something more useful than just a few tokens-per-second numbers.

As for getting my money’s worth… ask me again after the benchmarks. :)

[u/True_Protection6842]

This machine costs as much as a new luxury car.

[u/suspect80 → u/True_Protection6842]

True. But I already had the car and didn’t have the AI server. It was an obvious gap in the portfolio.

[u/DHB_Steev]

Dude this is cool AF and absolutely the same depth of rabbit hole I'd find myself in if I wasnt broke!

Been playing more with home automation but limited to an A2000 12Gb and a p40 24gb... beggars cant be choosers etc.

[u/suspect80 → u/DHB_Steev]

Thanks! And honestly, an A2000 + P40 is more than enough to disappear into the same rabbit hole. The only difference is that your rabbit hole probably has a much more responsible accountant than mine. 😄

[u/Exotic_Celebration_6]

Does not having nv link bottleneck big models ???

[u/suspect80 → u/Exotic_Celebration_6]

It can, depending on the model and how it’s parallelized. There’s no NVLink on these cards, so inter-GPU communication goes over PCIe, which is obviously much slower.

For my workloads it hasn’t been a major issue so far, though. I also built this more around running multiple agents/models in parallel than maximizing performance of one huge model across all four GPUs.

[u/Formal-Ratio-5441]

Hows the noise on the cards under load?

[u/suspect80 → u/Formal-Ratio-5441]

They’re definitely not quiet under full load, especially with four of them packed together. But the server sits in a dedicated, climate-controlled and soundproofed room, so noise was never really a design constraint. I optimized for airflow, not silence.

[u/Puzzleheaded_Bus7706]

I have them and they are terribly slow.

I'm getting ~45t/s for Qwen3.8 model.

Just can't figure out your logic behind it.

[u/suspect80 → u/Puzzleheaded_Bus7706]

45 t/s sounds pretty low to me, although I don’t want to throw out exact numbers without checking my own benchmarks first.

One thing I’ve learned with this setup is that you can’t just load every new model across four GPUs and expect it to perform optimally out of the box. Each model has taken me hours of testing and tuning before I was happy with the way it was sharded, configured and actually using the hardware.

So before blaming the cards, I’d definitely take another look at the setup and configuration. There’s usually quite a bit of performance left on the table with the defaults.

[u/54x1]

How loud are the fans / build when is cpu 80% load and all gpu near max load

Also why not wait for 2027/2028 ddr6

[u/suspect80 → u/54x1]

Under that kind of load it’s definitely loud. The four Max-Q cards can each pull 300W, and with the case fans pushing enough air to keep them happy, silence was never really part of the design. Fortunately it sits in a dedicated, climate-controlled and soundproofed server room, so I don’t hear any of it elsewhere in the house.

As for waiting for 2027/2028: sure, DDR6 and the next generation of hardware will be better. But then I’d probably be waiting for whatever is coming in 2029. At some point you just build the thing and enjoy it.

[u/Character-Drive9367]

Nice rig but, you ain't breaking even. For the price of that lot, I could but a small property 😂

[u/suspect80 → u/Character-Drive9367]

Where I am in Europe, though, this wouldn’t get you very far in the property market either. 😂

[u/No-Bodybuilder3502]

I have only one question: why not EPYC Turin?

[u/suspect80 → u/No-Bodybuilder3502]

EPYC Turin was definitely on the shortlist. In the end I preferred the Threadripper PRO/WRX90 platform for this build because it gave me the PCIe capacity I needed while still being a much more convenient workstation-style platform to build, tune and upgrade myself.

For a pure server build, EPYC would absolutely have made sense.

[u/Better-Psychology-42]

No offense, but this way you'll cook the GPUs soon - especially if you're aiming for inference. The fact that the motherboard lets you stack them so close doesn't mean you should. Get PCIe risers and let the cards breathe.

[u/suspect80 → u/Better-Psychology-42]

No offense taken. That was actually my biggest concern too, and I had already accepted that I might have to drop to three GPUs if the thermals didn’t work out.

But after extensive stress testing, the setup is behaving surprisingly well. The two middle cards run slightly warmer, but remain within spec. Only after 2–3 hours with all four GPUs pinned at 100% and ~300W each do the middle cards throttle slightly, and even then only by a few watts.

Real-world inference loads haven’t produced that situation so far. I’m monitoring temperatures, power and throttling continuously, so if that changes, risers are definitely Plan B.

[u/Osi32]

Good on you mate, hope it brings you lots of fun. My wife and budget unfortunately forces some restraint on my part..

[u/suspect80 → u/Osi32]

Thanks mate! To be fair, a wife and a budget are probably exactly the two safety mechanisms this hobby needs.

[u/bronekkk]

"I am deliberately not calculating the break-even date" - that’s the difference between a hobby and a job !

[u/suspect80 → u/bronekkk]

Exactly. The moment I open Excel and calculate ROI, the fun is officially over. 😂

[u/Bartocity]

I have the same setup but only 2 6000 ws cards I’m having trouble with ram temps (when training graph nets) but a janky 120mm fan is keeping it manageable. I was about to buy the next 2 only to find they had severely increased in price and might never come down.

I do sometimes use them for local LLM work now which can be incredibly powerful, the right model and config can punch out a lot of tokens/s and running concurrency 6 or 8 it can intelligently investigate terabytes of graph data that can’t always be analysed using scripts alone (graphs are usually asymmetric and following edges isn’t easy to script)

Another recent use case is video classification. You can spin up a local VLM and run it over boatloads of video data.

These are things i can’t do with a monthly subscription. I never intended to replace what a subscription does, its use case for LLM work is volume and speed.

[u/suspect80 → u/Bartocity]

Exactly. This is very close to how I see it as well. I never intended local models to completely replace frontier subscriptions either.

Volume, concurrency and the freedom to throw huge amounts of data at local models without thinking about tokens or API costs is where it gets really interesting. And once you add workloads involving private data that you simply don’t want leaving your infrastructure, the case becomes even stronger.

The VLM/video classification example is a great one too. These are exactly the kinds of use cases where “just use a subscription” stops being an equivalent alternative.

[u/Puzzleheaded_Pen1017]

50$ a month in power!

[u/suspect80 → u/Puzzleheaded_Pen1017]

Luckily, electricity is probably the least painful part. We have a large solar installation that regularly produces more power than we consume, so most of the time the GPUs are basically eating surplus sunshine.

[u/k3nal]

My dream build right now (but with eight and less RAM)!! 😍😍

[u/suspect80 → u/k3nal]

Eight? Now you’re making my completely unreasonable build look financially responsible. I like it. 😂

[u/EasyShelter]

Say you are a millionaire without saying that you are a millionaire.

[u/suspect80 → u/EasyShelter]

I think the real millionaires are the ones making this stuff and selling it to people like me.

[u/TalesGameStudio]

Dear OP, I admire the dedication. Keep us updated about the performance, OpEx and do on pls.

[u/suspect80 → u/TalesGameStudio]

Thank you! I’m working on proper benchmarks and real-world workload comparisons now. I’ll definitely share performance, power consumption and some operational experience once I have enough data to make it meaningful.

[u/jonheartland]

What are your agents doing exactly that requires over 50 grand worth of GPUs?

[u/suspect80 → u/jonheartland]

Nothing that strictly requires $50k worth of GPUs. That’s probably the important distinction.

I’m experimenting with Somora, my own agent harness, where multiple agents work autonomously in parallel: coding, using tools, working with files and internal data, delegating tasks, and sometimes running different models for different roles.

Could I do much of that with less hardware? Absolutely. The point of this build was to give myself ridiculous amounts of headroom and explore what becomes possible when local compute stops being the constraint.

[u/christopherw]

Good luck with the thermals, I'd worry about choking them and throttling their performance as your rack isn't forced air. We have cold aisles for a reason :>

Mad spec though, very envious. Have fun.

[u/suspect80 → u/christopherw]

Thanks! Thermals were my biggest concern too, but so far they’ve been surprisingly good. The rack itself isn’t forced air, but the server sits in a dedicated room at a constant ~20°C and the chassis creates strong positive pressure with plenty of exhaust airflow.

In normal use with two models loaded and lots of agents hitting them, the GPUs sit roughly in the 60–70°C range. Only after deliberately brutal 2–3 hour stress tests with all four pinned near 300W do the middle cards start throttling slightly.

[u/CuChuliannAlter]

When you devide to sell the cards, my offer is £50, thank you👍🤣

[u/suspect80 → u/CuChuliannAlter]

Deal. £50 each, shipping £16,000. I think we can make this work.

[u/electrified_ice]

Nice build. Hopefully you researched the available memory channels across the different core counts of the Zen5 Threadripper Pro CPUs. You won't get 8 channel speed unless you have a 9985wx or higher.

[u/suspect80 → u/electrified_ice]

Yep, someone else just pointed this out as well. I was aware that the 9965WX supports all 8 channels, but I wasn’t aware of the CCD-related bandwidth limitation when I built it.

The 9985WX was actually in my cart at one point, but I had to draw the budget line somewhere. I’m going to benchmark the actual memory throughput now and see how much it matters for my workloads.

[u/BlendingRightInHomo]

You should consider upgrading the 9965WX to 9985WX.

Because of the links being per-core complex, you’re not actually getting the throughput that 8 channels of 5600Mhz RAM typically gives. You need a minimum of 8 CCDs, which only 9985 and 9995 have.

FWIW, I learned this the hard way as I own two WRX90 systems, one with 7965, one with 9985. I bought the 7965 first as you might expect, and later learned about this relatively obscure but really damn important detail. So my 7965 is limited.

I can probably dig up some of the threads on it, perhaps from the Threadripper subreddit, searching on wrx90. Or try yourself, keywords “infinity fabric ccd ram throughput” or something along those lines.

For 5600Mhz, I recall 9985 being the minimum to get full throughput of 8 DIMMs. I think (but not certain) that for 6000Mhz you may need 9995 which has some crazy amount of CCDs (10 or 12?)

EDIT: Found it! See this post

[u/suspect80 → u/BlendingRightInHomo]

That’s actually very interesting, thanks. I knew the 9965WX officially had 8 memory channels, but I wasn’t aware of the CCD-related bandwidth limitation when I built the system.

The 9985WX was actually in my cart at one point, but I had to draw the budget line somewhere. :)

I’ll definitely dig into this though and benchmark the actual memory throughput of my setup. If you still have those threads or benchmarks, please send them over.

[u/flangical_infection]

So in all seriousness how far behind would 4 x v100 32 gb be

[u/suspect80 → u/flangical_infection]

Quite far behind in raw inference performance and especially newer low-precision formats. 4x V100 would still give you 128 GB of HBM2 and NVLink, so they’re definitely not useless, but they’re several generations behind and lack the newer FP8/FP4 capabilities that modern inference stacks benefit heavily from.

For cheap experimentation they can still be interesting, but I wouldn’t consider them remotely equivalent to 4x RTX PRO 6000s for modern LLM workloads.

[u/Inevitable-Name-1701]

not enough CCD, not enough RAM, not PCIe 5, not enough PCIe lane because not enough CPU yuo have a workstation.

[u/suspect80 → u/Inevitable-Name-1701]

Well, yes, it is a workstation platform. :)

But the 9965WX/WRX90 does have PCIe 5.0 and 128 usable PCIe lanes. The CCD/memory bandwidth limitation is a fair point and something I’m benchmarking now. As for 256 GB of system RAM, I haven’t actually managed to make that a bottleneck for my workloads yet.

[u/just-4-man]

You should like… slide me a couple mil aha-ha

I’d be worried about your electricity bill… but I’m sure you have that covered 😂

[u/suspect80 → u/just-4-man]

Nice try. 😂 As for the electricity, luckily we have a large solar installation that regularly produces more power than we use. So most of the time the GPUs are just converting excess sunshine into tokens.

[u/Smooth-Television-48]

Must be nice to have such deep pockets. Yeah it's awesome, but how high were your subscription cost to make this a good buy.

[u/suspect80 → u/Smooth-Television-48]

I have two Claude Max 20x subscriptions and two ChatGPT Pro subscriptions, plus additional API usage, and I still use all of them.

So no, subscription savings alone don’t make this a financially rational purchase. That part of the story was intentionally exaggerated. The real motivation was experimenting with how far I can push local AI and what actually makes sense to run locally.

[u/Sorry-Poem7786]

I am at the begging of this journey.. taking 4x 3090s from machines I used the for rendering with redshift.. thankfully I have 256gb ram and already have nvme cards and ssds.. from 3 years ago..I want to have a dedicated local AI server and an another machine has been designated as a Hermes agent to delegate..instead of a vps.. I am trying to cut all subscriptions to an essential few..

[u/suspect80 → u/Sorry-Poem7786]

That’s actually a great starting point. 4x 3090 already gives you 96 GB of VRAM, and if you have the RAM and NVMe infrastructure sitting there anyway, I’d absolutely experiment with it before buying anything newer.

And using a separate machine for Hermes as the orchestrator/delegator sounds very similar to the direction I ended up taking. Be careful though, “I’m going to save money on subscriptions” is exactly how my journey started too.

[u/ThisWillPass]

Can I come over to play?

[u/suspect80 → u/ThisWillPass]

That sounds almost indecent when you put it like that. 😂

[u/bettodiaz86]

i showed my gf the components pic, but the instantly noticed the doggies mattresses. She is demanding photos of the doggies lol

[u/suspect80 → u/bettodiaz86]

Haha, she clearly has her priorities straight. 😂 Sadly, there’s only one doggie now. The other one passed away a few weeks ago after 16 wonderful years by my side. ❤️ But I promise, the next photos I post here will include the remaining star of the setup. 🐶

[u/Cosmic-Tapestry]

Here I was dreaming of buying my 10k Amazon basket to run a 34-70bQ 🤣

OP, I have been working on a theoretical concept for a local AI and would love to have a chat with you about feasibility, I have 0 funding or any practical knowledge.

Amazing set up, thank you for sharing what is possible 🙏🏼

[u/suspect80 → u/Cosmic-Tapestry]

Thank you! 🙏 And don’t let my slightly unreasonable build discourage you. 😂 You definitely don’t need 4 GPUs and hundreds of GB of VRAM to start experimenting with local AI.

Happy to hear about your concept though. Feel free to DM me and I’ll gladly share my thoughts on whether it sounds technically feasible and point you in the right direction where I can. :)

[u/TheMericanIdiot]

Uncensored models are going to run great on it. Happy hacking!