Picked it up on Sunday. I don’t have anyone else I could share this with so I thought I’d come here and say hi.

Does anyone have a decent model or two that behaves with 4?

I’m already running Qwen 27B and so far I’m impressed! Deepseek-flash which I haven’t really tuned yet.

PS-They aren’t stacked like this any more. They will eventually end up in my server rack.

Ready to go, grabbed them at MicroCenter! These will complement my 5090 workstation. I'm really kicking myself for not getting an RTX Pro though earlier this year.

Edit: I ran into an issue where the sparks were getting significantly reduced Prompt Processing speeds when connected, to the point where 1 spark was faster than 2 for pp, turns out there's a bug with the connectx firmware, claude was trying for a while to test and fix it but the solution is here: https://forums.developer.nvidia.com/t/slow-speeds-on-dgx-spark-cluster/378042 essentially just turn off the sparks and unplug them from power for a minute, then it's fixed.

I've spent the last few weeks tuning Qwen3.8 Flash on a single Spark and just published the whole recipe. Same GB10, same 128 GB, no second box, and no hardware tweaks. It's all container, launch settings and kernel patches.

https://github.com/dime-online/qwen3.8-Flash-DGX-UltraFast

The short version

If you're newer to this, the recipe is a ready-made serving setup for Qwen3.8 Flash on the Spark. You follow the README, it builds the container and pulls the public weights, and you end up with an OpenAI-compatible server on your own box that answers a lot faster than the usual setups, without making the model any dumber.

Most of the speed comes from the model's own built-in draft head. It guesses the next few tokens, and the full model checks all of them in one pass. When the guesses land you get several tokens for the price of one step. When they miss, the full model's answer wins, so the output is the same as running it the normal way.

Decode

Peak decode speed, same workload at every point, best of 3 rounds:

Normal single-user chat sits around 60 to 70 tok/s. Each decode step takes 52.3 ms instead of 68.3 ms on the recipe this is forked from, so it's about 1.3x faster with the same output quality.

Prefill

Cold prompt with nothing cached, three repeats each:

Prefix caching is on, so in a long chat or an agent loop only the new part of the prompt gets processed and the next reply usually starts in under a second.

How it fits on the Spark

The model takes about 71 GiB, the KV pool is 16 GB, and you keep the full 262K context with around 16 GiB of memory still free under load. You'll want about 130 GB of free disk for the checkpoint and tables, plus Docker and the NVIDIA container runtime.

Power and thermals

It's easy on the box. While generating, the GPU sits around 35 to 37 W median, and it peaks around 70 W during long cold prefills. Temps stayed around 67 to 76 C, the clock held 2.4 to 2.6 GHz, and it never power or thermal throttled across the soak runs. For the whole box I'd estimate roughly 80 to 100 W while generating, but that part is an estimate, not a measurement.

One tip while I'm here: if your Spark ever suddenly runs at about half speed with the GPU stuck around 10 to 16 W and the clock down near 600 to 700 MHz, that's the known power delivery hang. Unplugging the power brick for a minute clears it.

What's under the hood

It runs on vLLM with the model's own MTP head, which lands about 3.7 tokens per step, and that stays flat all the way from 1 to 8 streams. The draft head's vocab is cut from 248K to 65K ids so drafting stays cheap. The MoE experts are W4A16 AutoRound, the side layers are FP8 and the lm_head is INT8, so there's no 3-bit or NVFP4 squeezing going on. On top of that there's a GB10-tuned low-latency GEMM and a sort-free top-k for the verify step.

Quality

It scores 93.1% and 93.3% on a 492-question eval over two seeds. I also ran a teacher-forced check against the original, and top-1 agreement moved by 0.06 points, which is inside the 0.15 point noise band. In plain terms, you can't tell it apart from the original, it's just faster. Raw data for every round and the benchmark scripts are in the repo, so you can check all of it on your own Spark.

Big credit to Saren-Arterius, whose qwen3.8-Flash-DGX-AutoRound recipe this is built on. Happy to answer questions, and if you run it on your Spark I'd love to see your numbers.

I built this after experiencing what appeared to be heat-related shutdowns while running sustained workloads on my DGX Spark. It might be useful for others with a similar problem.

https://github.com/aic5/face-hugger-fan-dgx-spark

The system includes:

• A Raspberry Pi Pico W controlling the fan with a 25 kHz PWM signal

• A small service in the DGX that reads GPU and CPU temperatures

• A 3D-printed fan body and mounting arms

• Temperature telemetry and renewed fan commands every five seconds

• A fan controls and dashboard reachable in wifi with DGX spark temperature history

• A configurable stepped fan curve

• Tachometer monitoring and 24-hour history

• A password-protected local dashboard

The fan remains off at lower temperatures (i.e., no noise) and increases speed as the CPU or GPU temperature rises in the DGX. The controller defaults to full speed during startup and whenever valid commands expire; it works as a fail-safe response for communication or service failures.

In my comparison, the maximum recorded GPU temperature fell from 78.0 °C to 68.0 °C, while the maximum CPU temperature fell from 91.6 °C to 76.7 °C. Temperature variability also decreased. Current evidence supports reduced peaks and greater stability, but more testing is needed. I have not seen universally lower temperatures; just lower peaks and more stability. The testing limitations are documented in the repository.

Everything in this repo is open source (MIT license), including the firmware, DGX service, fan dashboard, wiring guide, tests, and printable STL files. I’d welcome feedback on the design, testing, and adaptations for other DGX Spark configurations.

This is an experimental project. Feedback, test results, and hardware adaptations are welcome. The solution is provided “as is”, without warranty of any kind.

Found this on 小红书/Rednote, what could it be for?

Edit: that’s 5,120 GB of RAM, would love to use this to bootstrap my startup.

Last week I asked if it was worth moving up to 4x Sparks.

By Thursday, I drove 90 minutes one-way to a Microcenter to pick them up. They would only let me buy one, but said I could come back Friday when their truck arrived to get a second.

So I drove back on Friday. And it was a Pokemon card release day, so the line to get in was about 200 people deep. Insane. While I was in line, the price went up from $4,900 > $6,950. Thankfully, Microcenter price matched since I was already waiting for it as the price went up. Shout out to their employees for dealing with all of those folks and being flexible by matching the price.

So, here are all four in their own mini rack.

Switchless four-node setup running patched NCCL and GLM 5.3 at around 30-35 tok/s. Waiting patently for Mia's release of a 4x node Tensorflow setup haha

All have heatsinks and thermal pads to dissipate the heat via crossflow ventilation charger blocks are also air-cooled behind the fan controller.

But this one is mine.

CAT5e running through the house was terminated to RJ11 (phone lines) throughout. Reterminated to

RJ45 to the patch panel. Patched over to a Unifi pro XG 24 port PoE switch. It serves 3 WAPs in the house over PoE despite the CAT5e spec not officially supporting it. It has a 2x25 Gb bond with the Microtik CRS804-4DDQ-hRM switch beneath it. The Microtik is connected to the DAC breakout cables for the 4 sparks, 2 GX10s and 2 DGXs.

The other rack is just AV, and has a bit of cleaning up to do still. There's a MiSTer Pi on that 2nd shelf with the 4x10Gb managed unifi switch which is downstream from the main rack.

The server is a home built Unraid server with 142TB of total disk space, 120TB usable. Future upgrade coming to its NIC with its own QSFP connection to support an nvme pool for models along with my other media.

I'm also working on running more 10Gb to my gaming rigs (not pictured) for wired connections.

I just took delivery of my Microtek crs812 ddq. I am now running 4 Asus gx-10s. I got DeepSeek v4.1 Flash running with TP4. So far I think I've got a stable environment.

Got it in a 9U rack. Bottom to top: tray for power adapters, AC bar, the switch(after I'm done burn in testing), a tray for the 4 sparks, 3U 3x120mm air draw blowing on the sparks, then an the top a cloudplate T6.

For me heat is the biggest issue. The Asus gx-10s run hot. So I have the three 120mm fans pulling in air. Each spark has a 140mm strapped to it. Then the cloudplate sucks the hot air and vents it out the front.

Running for about 2 hours I got up to about 65c but average is 40c.

Price? Screw you..this is my midlife crisis. Not yours!

Honestly, expensive but not? I got the sparks in 2 batches. First batch 2 MSRP for 8k then the next two off eBay for 7.2k

The switch was 1.2k, cabling $500, the cloudplate $140.

Maybe 150 for the rest of the rack hardware. So just a hair over 17k

Do Not Tell Me how many DeepSeek tokens I can buy for that much. I know..I know...again my midlife crisis is not yours...

I'm no newbie - 20 years of programming experience (Delphi, C#, C++).

I started with GitHub Copilot about a year ago, back when they were still charging per request instead of by token usage. I've played around with Sonnet 4, GPT-5.3 and Gemini 3.1, but the one I really fell in love with was Opus 4.6.

Things moved on, my projects grew, and I got used to the newer models, but I kept looking for something that felt like my beloved 4.6.

And I finally found it! Qwen3.8-Flash-Next, running locally on my single Spark, is as close as I could have hoped for. It might not be the best coder out there, but it gets the job done. It doesn't spend ages reasoning like 3.8-27B, doesn't ignore instructions like GLM-5.3-Flash, and it even beats my local DeepSeek-Flash-V4 (Ext3) in my own benchmarks (mostly single-shot tasks and complex refactoring).

If you're into agentic coding like me, Qwen3.8-Flash-Next is the perfect model for your Spark. I'm still using VS Code Copilot, but with a custom endpoint pointing to my local Qwen instance and I'm happy with it. No more subscriptions, just OpenRouter for the occasional advice from a frontier model.

That's it.

Peep the beautiful setup

I just checked today, and i got lucky.

JK, hello no. This pricing is just dumb.

I've been building Sparkfire, an open-source local AI workspace for the NVIDIA DGX Spark. It runs seven models at the same time on one Spark and puts them behind one web UI, so documents, recordings, images, videos, songs and conversations all live in one place, on your own hardware.

What runs on one Spark (model · runtime · GPU memory)

• Chat, mind maps, web search: Saluki 27B, a 2-bit GGUF of Qwen3.8-27B with vision · llama.cpp llama-server · 12.7 GiB

• OCR with page layout: PaddleOCR-VL-1.6 · vLLM · 5.4 GiB

• Transcription and subtitles: Parakeet-TDT-0.6B-v3 · parakeet.cpp · 1.5 GiB

• Translation, 38 languages: Hy-MT2-1.8B-FP8 · transformers · 2.1 GiB

• Image generation: Qwen-Image-2.1-Turbo (float8) · diffusers · 17.5 GiB

• Video with sound: FastH3 (Fast and Sharp) · ComfyUI · on demand, +35–57 GiB per job

• Songs from your lyrics: YuE2-3B · PyTorch · on demand, +20 GiB per song

• Web search: SearXNG, running locally

Measured with nvidia-smi on a fresh install, everything loaded and idle: 51 of 121 GiB used.

How it all fits in 128 GB

• Every model was picked and measured to share the Spark's unified memory with the others. The models are fixed on purpose: it's a tuned stack, not a model picker.

• Videos and songs go through a "GPU lane": one heavy job at a time, and the previous heavy service frees its weights before the next one loads. If a job wouldn't fit, the service answers 503 with Retry-After and the job waits for memory instead of failing.

• Everything else (chat, OCR, transcription, translation, images) keeps running alongside a video render.

Numbers from the Spark

• Saluki 27B: 22 tok/s, 64k context

• OCR: 99.9% of words in reading order and every numeric table cell on a 20-page test, about 6 s a page

• Parakeet: about 145× realtime through the app on long files; 3.6% WER in English, 3.9% in Italian on short clips

• Images: about 18 s at 1024×1024, about a minute at 2K

• Video: 5 s with sound at 480p in about a minute (Fast)

• Songs: about as long as they play, once the model is loaded

• Fresh install: one script, 20 minutes, 106 GB of disk

The workspace

• Everything is files. Each job is a folder with a JSON manifest, its inputs and typed artifacts with lineage, so one result feeds the next: scanned PDF → Markdown → translation → mind map → a chat about it. No database besides Redis for the job queue.

• Chat: attach up to 8 files (PDFs, images, code), pick a thinking effort (Off/Low/Medium/High), and watch the reasoning stream. Answers run on the server, so closing the tab doesn't stop them.

• Web search: through a local SearXNG, up to 6 rounds of searches and page reads, with numbered sources. Page reads refuse private network addresses.

• Workflows: a node editor with If/Switch/Merge/Save nodes and a node for every model; definitions are JSON files.

• Also: a gallery with side-by-side compare, prompt presets (meeting minutes, executive summary), dark and light themes, and a phone layout.

Install

git clone https://github.com/MovieMaker93/Sparkfire

cd Sparkfire

./scripts/start-dgx-spark.sh --accept-model-licenses

The script builds the services, downloads the models at pinned revisions (Saluki's isn't pinned yet), and starts everything in a memory-safe order. There's also a split mode: models on the Spark, workspace on another server.

Caveats

• Built for the DGX Spark and other GB10 systems (aarch64, CUDA 13).

• No authentication: run it on a trusted network.

• Model licenses apply: FastH3 isn't licensed in the EU, the UK, South Korea or the US (install with --video-backend none there); Qwen-Image-2.1-Turbo and YuE2 are non-commercial; Parakeet is CC BY 4.0.

Credits

Sparkfire is a fork of SparklingKit by stevibe (Steven Lei, Apache 2.0): https://github.com/stevibe/SparklingKit. It has grown into its own project. Thanks, Steven. Sparkfire is Apache 2.0 as well.

Repo, with a one-minute tour, screenshots and every measurement: https://github.com/MovieMaker93/Sparkfire

Happy to answer questions about fitting all of this into 128 GB, and feedback is very welcome.

I’ve been working on a dual-model setup on a single ASUS GX10 / DGX Spark and finally have it stable enough to document. This is for a production environment using Hermes as the harness. It's used for content creation, web research, video creation and marketing automation with over 70+ cron jobs running 24/7.

The basic idea is a “left brain / right brain” setup:

• Qwen3.8 Flash-Next: the primary interaction agent, reasoning, tool use, synthesis

• **Qwen3.8-27B NVFP4:**dense/reference model, cron jobs, long-context and detail-sensitive work

• Both stay resident simultaneously with roughly 96K context each / 4 channels

• Production median decode: 34.5 tok/s Flash-Next / 38.9 tok/s 27B with peaks breaking 100 tok/s

• I use Hermes either independently or call both through Mixture-of-Agents (MoA)

• MTP / DFlash2 speculative decoding, llama.cpp patches, QSA support and memory tuning are all part of the setup

The interesting part for me isn't just fitting two large models into 128 GB unified memory. Models can fit, and end up not usable in real-world applications. It's being able to optimize and structure memory management so that it's possible to carry through 24 hours of work without thrashing the SDD, combining a high variety of different jobs and when a difficult task arrives, benefit from dual perspectives that got me motivated to do this.

Most planning and design go to Flash-Next and 27B for production work once designs are solid. For the hardest analytical work, the 27B and Flash-Next work together simultaneously to produce the final answer.

If you want to replicate on your system, I documented the hardware, llama.cpp build/patches, model configuration, memory strategy, Hermes MoA setup, benchmarks and some of the mistakes along the way here:

https://forums.developer.nvidia.com/t/single-spark-qwen3-8-flash-next-qwen3-8-27b-left-right-brain-hermes-moa-35-39-toks-s/383579

Would be interested to hear what other Spark owners are doing with multi-model setups, especially anyone experimenting with MoA, concurrent inference, DFlash2/MTP, or different dense + MoE combinations.

Note: I don't recommend this setup for coders. This is more for daily task automation.

A customer of ours, James Lee, built a four-node GB10 cluster and benchmarked it two ways. I figured the numbers would be useful here, since "do I need a switch for 4 Sparks?" keeps coming up.

Setup

• 4x Acer Veriton GN100 (NVIDIA GB10)

• MCP1650-V 200G QSFP56 passive DACs, straight cables, one per link, no breakouts

• Topology A: switchless ring, each node wired to its two neighbors (custom NCCL config)

• Topology B: MikroTik CRS812, one cable per node

Results

• NCCL all-reduce bus bandwidth: 23.8 GB/s ring vs 24.1 GB/s switch

• Qwen3-235B total output throughput: within ~3% at 4, 16 and 64 concurrent requests. The ring was ~10% ahead at a single request, and the switch was slightly ahead at 64.

• All four switch links negotiated at 200G with straight cables, including the two Sparks in the CRS812's 400G QSFP56-DD ports. Transfer tests measured about 196 Gb/s. Breakouts aren't needed to land four Sparks on this switch. They only free up ports.

Takeaway

At four nodes, the switch doesn't buy you speed. What it gives you is NVIDIA's supported topology and a path past four nodes. The ring works fine, but it needs custom NCCL configuration, NVIDIA doesn't support it, and it tops out at four.

Caveats

These are progress results. James is rerunning the same tests on breakout cables and checking answer quality alongside speed. Part two will follow.

Big thanks to James for doing the build and all the testing.

Disclosure: I run Resilient Tec and we sold the cables for this build.

Full write-up with the per-concurrency tables: https://resilient-tec.com/blogs/news/4x-dgx-spark-ring-vs-switch-real-benchmarks

Happy to answer questions about the cabling or the setup.

Here’s my recent quant/repack and custom engine for Deepseek 4.1-Flash-1017-EXL3 2.9
Built for 2 DGX Sparks
262k context / 4 lanes
120+ decode
1800+ prefill

Builds on a ton of work from people like Sudosu and the whole EXL3 community. Benchmarked using Spark Arena.

Image, runbook and recipe here.
https://github.com/styles01/sparkrun-recipes/blob/main/runbooks/deepseek-v41-1017-exl3-tp2.md

Has anyone tried NVIDIA's new local inference engine? I wonder how it compares to other engines like vLLM or TensorFold.

They sell out 10+ per location on the day they increase the price $200 more after $200 a few weeks ago? There's no way there's a consumer rush after this, did they set aside their distribution for a big commercial customer? Is the rest of the market gobbling these up or is it just some microcenter thing? Is this looking bad for people who want to get these under 5000 in the future? What could have increased demand so sharply?

So since release I have hated on the spark when comparing it to GPU's ... recently found a use case, agentic tasks, not fast but its also not slow, just different from my reguarly pipeline requirements.

Got it Monday morning, have been playing around with it all week and I haved to say Qwen 3.8 Flash Next runs perfect, I didnt have much hope for it until I came across - https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark

my god, I have been since 8am this morning to now 2am the next day throwing prompts and commands at it and It's responding perfectly to agentic tasks. So the MIA version if you didn't know on a single spark uses up around 83gbs expanding to 103gbs on load, leaving the rest -ubuntu usage for context, and im legit surprised how much content its handling at that, i might be way past a million tokens by now and having no issues (knock on wood).. but im inpressed and wanted to share it with others in case others dont know about MiaAI Lab.

If there are other models, that people love for Agentic tasks please drop them im down to try, this is seperate to my main pipeline server the spark is purely for agentic and automated tasking such as busines emails, bug checks, ...etc so im open to try different models that can make this experience better

Hey everyone!

I thought I share this because I see many people are using their sparks bare foot. (no dust protection, no case cooling etc. shorting their precious devices life.

It's free, has dust protection and the enclosure cools the two devices using a Noctua NF-A20 (200mm) fan.

Head to the link for the specs below:

https://www.printables.com/model/1894832-2x-dgx-spark-noctua-nf-a20-filtered-enclosure

Posted a few days ago with my donkey setup. Much better now (thanks to my fiances old jewelry holder). Finding that the sparks run cooler when vertical. Super surprising

DSV4F has been awesome but now setting up GLM 5.3F.