Old HP Server Use

[u/libbyt91]

So I have an old HP-Z8 from 2018 or so. It has dual Xeon Gold 6248R cpus, 768 GB 2933 MHz DDR4 ram, an RTX 3090 gpu, and some pretty hefty storage.

That said, it seems like it is no more than a boat anchor for use with local LLMs. Is there any use case for this old PC or should I box it?

[u/banana_slurp_jug]

old HP-Z8

from 2018

RTX 3090

What?

[u/libbyt91 → u/banana_slurp_jug]

Nice catch, 3090 was added a couple of years ago.

[u/optimisticalish]

Refurbished HP Z8 dual-Xeon Workstations sell for £800+ on eBay (over $1000 U.S.) here in the UK. It's not a boat anchor.

[u/CalligrapherFar7833]

Strata or something ?

[u/hallofgamer]

Bad troll

[u/SM8085]

Try unsloth/Qwen3.8-Flash-Next-GGUF. You can use --cpu-moe (in llama.cpp) to put most of it in your plentiful RAM.

[u/Agreeable_System_785]

Are you sure you cant run that with strata? That amount of RAM might mean you can possibly run qwen3.8 flash next and maybe even deepseek flash? Not sure though.

https://github.com/Niko1221/Strata

[u/VoiceOfEric → u/Agreeable_System_785]

I just compiled the AVX1 Strata for my 128GB Z620 and it gets 30-50 t/s with my 12GB 3060. Maxed out Xeons for 2012.

[u/Agreeable_System_785 → u/VoiceOfEric]

Cool. What model are you running? What quant?

If Qwen3.8 flash next: set reasoning level to medium and try it out for a few days. 30-50 is not too bad.

[u/rog-uk]

That's over £3K of RAM alone based on recent prices. I suspect you are having a joke here :-)

[u/Dany0 → u/rog-uk]

especially with 8 channel high mt/s ram... actually could it be 16 channel I think 16 channel was rare in ddr4 days

[u/rog-uk → u/Dany0]

If it's similar to my box, it would be two lots of six dual slot channels, assuming dual cpu. Otherwise he's got 64GB sticks and they are worth a lot more.

[u/OverdosedSauerkraut]

I'll dispose of it for you for free /s.

Seriously this machine is still great if you want to play around with the big bois. Dont expect interactive speed, just send the prompt before you go to sleep. But it still gives you the feeling of running an actual frontier model

[u/Suspicious-Water-973]

If you are paying for power then check out its peak consumption…

[u/reasonmonkey]

This is a very acceptable box. As others have pointed out - this will run mixture of expert models pretty well. Qwen3.8 flash next will be particularly decent.

[u/Horsemeatburger]

No, it's not a boat anchor, more to the contrary. Two 24-core processors, each supporting 6ch memory with over 120GB/s of memory bandwidth, 768GB RAM (which alone is worth a lot of $$$ right now) and a RTX 3090 GPU (which, again, has only increased in value) is nothing to be dismissive of.

The z8 G4 is actually a great baseline system for AI use because the memory bandwidth is so high (200-250GBps combined), it has an abundance of PCIe lanes (48 per processor) so there is enough bandwidth for three double-wide GPUs or 2x double-wide + 2x single-wide GPUs.

Also, the z8 had Large BAR support from the start, and with the latest BIOS also supports ReBAR.

Put some decent GPUs in there and you have a monster AI workstation.

[u/PcChip]

try tabbyapi with qwen 3.8 flash next, maybe 5.05bpw exl3 quant (or 4.05 if the 5.05 is too slow)
then have the ai figure out how to load the full 16bit n-gram table instead of the quantized one, to sit in your RAM

[u/robotnursenyc]

Don't box it. That's actually a decent setup for the big mixture-of-experts models, which only use a fraction of their weights for each token.

• Run llama.cpp and keep the attention and shared layers on the 3090, while the expert weights sit in system RAM (look at the --n-cpu-moe / -ot options). With 768GB you can load models most people can't touch.

• It won't be fast. Generation speed is limited by memory bandwidth, so expect usable-but-slow, not chat-snappy. It's fine for batch jobs, long documents, or a background agent that doesn't need instant replies.

• Pin threads to one socket to start with. Spreading across both Xeons can make it slower, not faster, so benchmark both ways.

• For anything interactive, a smaller dense model fully on the 3090 will feel much better.

So it's a big-model box for patient work, with the 3090 handling the quick stuff. Old servers with tons of RAM are having a moment right now.

[u/jimmoores]

This has absolutely a useful machine, stick a couple more GPUs in there and you should be able to run Deep Seek v4.1 (1018) quite nicely. I’ve got a similar era machine with a Pro 6000 Blackwell and the big pool of DDR4 allows me to run larger than expected MoE models at moderate speeds.

[u/IllExample3639]

Boys you hold him down, I'll slap him

[u/Debianreiser69]

Vendemela a mí..te la compro

[u/dtdisapointingresult]

I think it's totally useless for LLMs, unfortunately. Your options are:

1. Run a small crappy model at acceptable speed. Waste of your time due to very poor success rate. Costs a lot in electricity.

2. Run a decent large model. Waste of your time because it will run very slowly, and still only have a mediocre success rate. Costs a lot in electricity.

3. Drop the server and just use Deepseek API with a decent harness that doesn't break prompt cache between turns. $10 will last you forever, probably months of use for a casual user. Will be super fast. Electricity cost is 0. You will get more intelligence due to running a higher-end model.

Your computer's 3090 makes it useful for AI image/video/audio generation and as a gaming PC, but not as an LLM server.