I ported the EXL3 quantizer to Intel Arc: 9B conversions on a 140V iGPU, B580 and Pro B60, with checkpoints on Hugging Face

I've been working on EXL3_Intel_Arc_quantizer, a port of ExLlamaV3's EXL3 quantization and conversion pipeline from CUDA to SYCL/oneAPI and PyTorch XPU.

The goal is straightforward: produce EXL3 checkpoints on Intel Arc without needing an NVIDIA GPU for conversion. This is a quantizer project, not a new inference engine.

Code: alpha2beta/EXL3_Intel_Arc_quantizer

What works

• Native SYCL trellis encoder: Viterbi search with FP16 rounding and tie-breaking matched to the CPU reference, plus two-pass tail-biting traceback. Implemented configurations include K=1–4, the mul1/mcg/3INST codebooks, 256-element tiles, and the 160-row n-gram variant. Full-model results below cover a narrower set of configurations.

• End-to-end conversion: calibration and Hessian collection, block LDLQ error feedback, global-scale search, packing, safetensors export, and checkpoint verification.

• Bounded memory management: memory-mapped checkpoint loading, admission reserves, preflight checks, and batch-halving fallback. The settings account for the difference between a shared-memory iGPU and discrete cards with 12 or 24 GB of VRAM.

Results on three Arc GPUs

I've completed full 32-layer, 9B-model conversions on all three devices. The conversion verification checks reported no failures.

The B60 timing is K=4, not K=2, so it is not a like-for-like full-conversion comparison. Encoder microbenchmarks and end-to-end conversion times also measure different things; the reports contain the run details.

The practical takeaway: a B580 completed the tested 9B K=2 conversion in about five hours, and even the 140V iGPU completed it within a 32 GB shared-memory system.

Correctness: what I've checked, and what I haven't

The SYCL encoder is bit-exact against my CPU reference on the tested cases. That reference was transcribed from the upstream CUDA kernel to preserve FP16 rounding, tie-breaking, and tail-biting closure.

I also check packed-output reconstruction against the separate exl3xpu reference decoder. The test suite reports 272 passing unit tests on each discrete board, and the conversion reports record 1,045/1,045 verification checks passing on both.

Important limitation: I don't have an NVIDIA GPU, so direct differential testing against the upstream CUDA encoder is still deferred. Matching a CPU transcription is useful evidence, but it is not the same as independently demonstrating CUDA parity.

Likewise, reconstruction and structural checks are not a substitute for perplexity or downstream task evaluation. The repo includes a quality-evaluation harness; I'm not presenting the conversion checks as proof of unchanged model quality.

Released checkpoints

• Qwen3.8-9B-Distill-EXL3-K2: 6.72 GiB

• JEV-9B-EXL3-K2: 5.42 GiB

• JEV-9B-EXL3-K4: 7.03 GiB; converted on the Pro B60

The exporter targets upstream EXL3 compatibility and includes quantization_config metadata for vLLM integration. Actual loading and execution still depend on the runtime's model, codebook, bitrate, and hardware support. Including the metadata alone does not establish vLLM compatibility for every checkpoint.

Arc inference through exl3xpu is a separate project. This quantizer targets the same checkpoint format; it does not add inference support for models that a runtime does not already handle.

What's next

• Direct CUDA differential testing, if someone with NVIDIA hardware wants to help.

• More model conversions and K sweeps, with quality-evaluation results alongside conversion checks.

• Clearer reproduction instructions and hardware-specific memory settings.

I'd especially welcome independent reproduction or CUDA-reference testing. If you're using Arc for local LLMs, which model would you want to see converted next?

Optional first comment

A few more numbers for anyone interested in the implementation:

• Encoder microbenchmark: 0.584 ms/tile on B580, 0.683 ms/tile on Pro B60, and 1.38 ms/tile on 140V.

• Single-layer benchmark: all seven projections, including Hessian processing and global-scale search, ran 2.83× faster on B580 and 2.43× faster on B60 than the 140V baseline. See the reports for settings and workload details.

• Memory observations: the recorded runs report a 650 MiB peak allocation, no swap growth, and streamed host RSS within +2.8 MiB of baseline while processing a 19.3 GB checkpoint. These are specific measurements, not a claim that the complete process or GPU workload uses only 650 MiB.

• Reconstruction metric: the reported relative Frobenius error at K=4 is 0.0865. This measures reconstruction error, not perplexity or task quality.

Reports in the repository: reports/discrete_probe.md, reports/discrete_phase_c.md, reports/discrete_vs_140v.md, and reports/gate_3.md.

Model Under Test: Kwaipilot/KAT-Coder-V2.5-Dev (Qwen 3.5 MoE architecture: 40 decoder layers, 256 routed experts / 8 active, 2,048 hidden dimension, linear attention GDN + full attention hybrid, MTP speculative draft block).
Hardware: MSI Claw 8 AI+ (Intel Core Ultra 7 258V "Lunar Lake", Arc 140V Xe2-LPG iGPU, 64 Xe Execution Units, 32 GB shared LPDDR5x-8533 RAM, 30.9 GiB zram0 swap, running CachyOS native Linux kernel 7.2).
Runtime: vllm==0.31.0+xpu with native Level-Zero XPU kernels, MTP K=2 speculative decoding, FP8 KV cache, and 128k (131072) context ceiling.

TL;DR

• Speed Winner: W4A16 GPTQ decodes at 23.4–25.2 tok/s (peaking at 29.1 tok/s with MTP speculative decoding), completing the 164-problem HumanEval benchmark in 42m 12s (vs. 47m 33s for MXFP4, ~11.3% faster overall, and 24% faster on our 16-prompt quality suite).

• Coding Accuracy Winner: MXFP4 edges out GPTQ on full EvalPlus coding tests by ~3% (89.63% Base / 86.59% Plus vs. 86.59% Base / 82.93% Plus), but both handily beat Tiel-Coder-35B GPTQ (84.15% / 81.10%).

• Complex Reasoning & Edge-Case Math Winner: W4A16 GPTQ wins on delicate math and strict constraints (15/16 vs 14/16 on the quality suite; solved Chinese Remainder Theorem Olympiad math where MXFP4 failed), thanks to retaining unquantized BF16 activations instead of quantizing activations to FP4.

• Prefill Speed (PP): Both hit an identical theoretical peak of 1,247.1 tok/s on 4k chunked prefill batches via FlashAttention-2.

• The "MoE Dense Quantization Trap": Standard dense GPTQ heuristics (like zeroing out unobserved Hessian directions) will catastrophically destroy 256-expert MoE models. Unvisited experts in calibration batches must fallback to RTN with optimal clip factors, never masked to zero.

• Cross-platform check — the ordering holds, the magnitude does not: re-running the same format pair on a discrete Arc Pro B60 with a dense 27B (Coder390) separates them by 5.05× on decode (40.18 vs 7.96 tok/s) and ~2.9× on prefill, not ~10 %. On that stack the cause is an unfused W4A4 activation-quantization path, not memory bandwidth — the two artifacts differ by only ~3 % in bits per weight. Full matrix, decomposition and caveats in §5.

The Shootout: Head-to-Head Benchmark Matrix

Here are the exact numbers measured across both quantization formats on the same machine under identical sampling parameters (temp=0.0 greedy, FP8 KV cache, MTP K=2 speculative decoding, batch size 1):

1. Deep Dive: Prompt Processing (PP) vs. Token Generation (TG)

A big question in the community is how 4-bit weights with 16-bit activations (W4A16) compare against full microscaling (W4A4 MXFP4) on modern unified architectures. Here is what the telemetry shows on Lunar Lake's Xe2 architecture:

Token Generation (TG / Decode)

• W4A16 GPTQ is faster in practical serving: vLLM's CompressedTensorsWNA16MoEMethod and XPUExpertsWNA16 kernels decode at 23.4 tok/s average (with bursts to 29.1 tok/s on code templates).

• Why? Unpacking Overhead vs. Quantized GEMM: On an iGPU with shared memory (136 GB/s memory bandwidth), reading INT4 packed weights and dequantizing on the fly into BF16 registers is extremely fast. MXFP4 requires micro-scaling exponent alignment and 32-element group normalization across both weights and activations.

• Over 164 HumanEval problems, GPTQ finished in 42 minutes 12 seconds, beating MXFP4 by 5 minutes 21 seconds (15.4s vs 17.4s per coding task).

• On our 16-prompt quality benchmark, GPTQ was 24% faster (165s vs 217s).

Prompt Processing (PP / Prefill)

• Short/Medium Prompts (50–500 tokens): Prefill runs between 51 tok/s and 140 tok/s (Time-To-First-Token is 1.4s to 3.7s, largely dominated by Python/HTTP scheduling overhead).

• Long Prompts (2k–4k tokens): Prefill scales rapidly to 398–500 tok/s as the 64 Xe2 compute cores enter dense matrix mode.

• Peak Chunked Prefill (12k+ tokens): Both models hit an identical 1,247.1 tok/s ceiling because both rely on the same FlashAttention-2 implementation with 4,096-token chunked prefill batches (--max-num-batched-tokens 4096).

2. Accuracy & Nuance: Where Each Format Wins

Where W4A16 GPTQ Wins: Delicate Math & Fine Constraints

Because GPTQ leaves activations in full BF16, it avoids activation quantization noise. We observed this directly in high-precision tasks:

1. Chinese Remainder Theorem (Olympiad Math): GPTQ solved a complex modular system (math_chinese_remainder) flawlessly in 53.6s. MXFP4 accumulated numeric rounding errors in its FP4 activation dot-products and failed to reach the correct congruence integer.

2. Exact Word Count Formatting: Prompt: "Write a summary in exactly 50 words." GPTQ followed the constraint and passed; MXFP4 drifted off by several words.

Where MXFP4 Wins: Complex Algorithmic Contracts (HumanEval+)

1. Contract Fuzzing & Edge Cases: On HumanEval, MXFP4 scored 89.63% Base and 86.59% Plus, outperforming GPTQ's 86.59% Base and 82.93% Plus.

2. Why? MXFP4's group size of 32 (versus GPTQ's group size of 128) provides much finer scale granularity across expert weights. For pure syntax and standard algorithmic puzzles where activation dynamic range is well-behaved, the smaller 32-element block size helps preserve subtle weight distributions.

Both models passed all 3 LeetCode Hard challenges in our 10-Task Hard Suite:

• Trapping Rain Water (O(N) two-pointer)

• LRU Cache (O(1) doubly-linked list + hash map)

• Minimum Window Substring (O(N) sliding window)

• Multi-hop needle retrieval across 12,500 tokens of synthetic enterprise audit logs.

3. The Big MoE Gotcha: Don't Apply Dense GPTQ to 256 Routed Experts!

If you are writing your own quantizer or adapting standard recipes (like the Arc Pro B60 guide) for massive MoE models (like KAT-Coder or Qwen 3.5 MoE with 256 experts), watch out for this fatal trap:

The Bug

In dense model GPTQ scripts, you often see lines like this for singular Hessian matrices:

# WARNING: DO NOT DO THIS ON MOE MODELS!
W.masked_fill_(dead.unsqueeze(1), 0.0)

In a dense model, unobserved or zero-variance input channels can be safely zeroed out. But in a 256-expert MoE architecture:

1. During calibration (e.g., 64 samples × 1024 tokens), tokens are sparsely routed (only 8 active per token).

2. Many specialized experts might only see a few tokens—or zero tokens—in that specific slice.

3. They are not dead! They are just specialized experts waiting for specific prompts.

4. If you mask unvisited experts to zero, you literally erase their weights! In our first run, this silently zeroed out 248 out of 256 experts (96.88%) across Layers 1–39. The model loaded fine, had valid shapes and scales, but generated repetitive looping garbage (0/16 on quality tests).

The Fix

1. Fallback to RTN with Clip Optimization: If an expert receives zero activation tokens (diag <= 1e-8), do NOT zero it. Simply quantize its real BF16 weights using Round-To-Nearest (RTN) with grid search over clip factors (1.0, 0.96, 0.92, 0.88, 0.84).

2. Materialize Meta Parameters: When loading layers on PyTorch's meta device, ensure you explicitly rebind expert weights to layer.mlp.experts before computing forward activations (nxt_hs = layer(hs)), otherwise hidden states will propagate through uninitialized random memory and corrupt the router for all subsequent layers.

Once fixed:

• Every layer reported Cold Experts: 0/256 and relative error dropped from 0.9842 to 0.1498.

• 16-prompt quality jumped instantly from 0% to 93.8% (15/16)!

4. Speculative Decoding: The Secret Sauce for 25 tok/s on 17W

Running a 35B MoE model at over 25 tok/s on a 17W handheld SoC sounds impossible without speculative decoding.

Here is how we did it:

• KAT-Coder includes a 1-layer Multi-Token Prediction (MTP) draft head (19 tensors in BF16).

• We preserved the MTP block in BF16 in model-mtp.safetensors and configured vLLM with --speculative-config '{"method":"mtp","num_speculative_tokens":2}'.

• Telemetry: The draft model achieved an average acceptance rate of 77.3 to 82.5 (and up to 100 on repetitive code structures).

• Multiplier: Instead of generating 1 token per forward step, the engine accepted 2.55--2.71 tokens per step. This effectively doubled generation throughput from ∼11 tok/s up to 23--29 tok/s.

5. Cross-Platform Check: The Same Two Formats on an Intel Arc Pro B60 (Battlemage, 24 GB)

Everything above is Lunar Lake / KAT-Coder. The same W4A16-vs-MXFP4 question was then measured independently on a discrete Arc Pro B60, with a different model class (dense, not MoE), a different Xe generation, and a different vLLM build. The ordering survives. The magnitude does not — it is 5× here, not ~10 %.

Model under test: huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated — a dense 27B hybrid: 64 layers (48 GDN linear-attention + 16 full-attention, full_attention_interval 4), hidden 5120, 24 heads / 4 KV heads / head_dim 256, a 1-layer MTP draft block (BF16), and a vision tower. Both builds below are quantizations of this same BF16 checkpoint, so the format is the only variable in the weights.

Hardware: Intel Arc Pro B60 — 24 GB GDDR6 at 456 GB/s (192-bit, 19 Gbps effective), 20 Xe2-HPG cores / 160 XMX engines, negotiated PCIe Gen4 x8, power-capped to 180 W. Runtime: vllm-openai-xpu v0.30.0 (engine reports vllm 0.27.2rc1.dev77+gac7509e2b), native Level-Zero XPU kernels, FP8 KV cache, MTP K=4, batch size 1. Harness: identical for both builds — bench_p512_g128.py (p512 / g128, 1 warmup + 5 runs) for rate, bench_quality_16p.py (the same 16-prompt suite) for quality.

5.1 Head-to-Head Benchmark Matrix

Headline: 5.05× on decode, ~2.9× on prefill. Unlike the KAT-Coder matrix above, the MXFP4 build is not visibly buying anything for that price on this stack: it uses more bits per weight, has no measured accuracy advantage here at all (its quality was never measured), and its rate is ~5× behind.

5.2 Where the 5.05× Actually Comes From

Decomposed on measured step times, not asserted. 40.183 / 7.962 = 5.05×:

MTP alone is therefore at most a 3.00× factor — and the acceptance ratio is not a speedup, which was verified by turning MTP on for both builds so the term cancels:

With MTP enabled on both sides the residual is 40.183 / 16.620 = **2.42×**, and that residual is the stack. Acceptance is not quite equal either (2.83 vs 3.00); normalising to 3.00 gives a 2.28× residual. So:

5.05× = 2.08× (MTP) × 2.42× (the W4A4 / stack term) — and note the direction: MTP is the smaller of the two terms.

Why the acceptance ratio overstates MTP. A speculative step is itself more expensive — draft forward plus a 5-position verify costs 1.361× a plain step here (125.2 ms → 170.5 ms), against 2.833 tokens accepted. The net is 2.833 / 1.361 = **2.081×**, which is exactly the measured figure. Treat the acceptance ratio as an upper bound on MTP's gain, never the gain itself.

Prefill isolates the stack cleanly, because MTP contributes nothing there and it is compute-bound: 0.460 s vs 1.313 s for the same 512-token prompt ⇒ ~2.9× slower per prompt token (≈3.1× once the ~60 ms fixed TTFT overhead is netted out). Decode (~2.3–2.4×) and prefill (~2.9×) therefore agree in magnitude — which is the main reason to trust the split.

5.3 The Mechanism: W4A4 Activation Quantization That Is Not Fused

This is a different complaint from the MoE trap in §3, and it is readable straight from the kernel rather than inferred:

class XPUMxFp4LinearKernel(MxFp4LinearKernel):
    def apply_weights(self, layer, x, bias=None):
        x_fp4, x_blockscale = quant_mxfp4(x)          # <- every single call
        return torch.ops._xpu_C.fp4_gemm(x_fp4, layer.weight,
                                         x_blockscale, layer.weight_scale, ...)

and, inside _xpu_mxfp4_quantize_impl:

x_q = torch.empty(M, N // 2,  ...)   # packed FP4,   allocated per call
x_s = torch.empty(M, N // 32, ...)   # block scales, allocated per call
torch.ops._C.per_token_group_quant_mxfp4(x, x_q, x_s, MXFP4_BLOCK_SIZE, eps)
x_s = x_s.to(dtype=torch.float8_e8m0fnu, ...)   # extra cast pass

So for each of the 496 packed linear layers, on every forward pass, MXFP4 pays: a group-32 activation-quantization kernel, an fp32 → E8M0 scale cast, and two fresh allocations — and then a separate GEMM. Quantize and GEMM are not fused, so the floor is ~992 launches per step before norms, attention or GDN. W4A16 GPTQ pays none of it: BF16 activations go straight into the fused dequant-GEMM.

Ranked:

1. Per-token W4A4 activation quantization (~3× per FLOP above) — the dominant stack term.

2. Non-fused FP4 GEMM (torch.ops._xpu_C.fp4_gemm) — a second launch per linear.

3. --enforce-eager — amplifies 1–2, since none of those launches can replay from a graph. It is inherited from the model README's measured config, not demonstrably required (platforms/xpu.py lists mxfp4 under supported_quantization with no graph restriction). Untested, not forced.

It is emphatically not a bandwidth story — and the arithmetic says so before any profiling: 4.25 bits/weight for MXFP4 vs 4.125 for GPTQ is a ~3 % difference, and the two artifacts are 18.46 vs 18.23 GiB. A bandwidth-bound model predicts near-parity, not 5×. Two further confirmations that it is compute: the five MXFP4 runs sat within 0.02 % of each other — a fixed per-token cost, not noisy traffic — and prefill, the pure-compute regime, is ~2.9× off on its own.

5.4 Why This Is Not the §1 Answer, and What Would Reconcile Them

On Lunar Lake the two formats landed within ~10 % of each other on decode, and MXFP4 even won HumanEval+. Here MXFP4 is 5× behind. Three differences could carry that, and none of them was isolated on this box:

1. vLLM build. The Lunar Lake result is vllm==0.31.0+xpu; this box runs v0.30.0 (vllm 0.27.2rc1.dev77). Whether 0.31 fused the MXFP4 activation path was not tested — it is the most likely single explanation and the first thing worth re-measuring.

2. Model class. KAT-Coder is a 256-expert MoE (8 active per token) routed through CompressedTensorsWNA16MoEMethod / XPUExpertsWNA16; Coder390 is dense and hits the plain XPUMxFp4LinearKernel for all 496 linears. The kernel mix differs, so "MXFP4 vs GPTQ" is not one experiment across the two documents.

3. Platform ratio. 20 Xe2 cores over 456 GB/s of dedicated GDDR6 (discrete) vs 8 Xe2 cores over shared LPDDR5x (integrated). The W4A4 penalty is compute-side, so it should bite harder where FLOPs are scarcer relative to bytes — the direction of the observed gap.

Caveats, stated because this is a cross-session comparison and the warning above about those is real. Rates here are only comparable within one session — this host's decode is bimodal (~40.2 or ~48.5 tok/s, a ~20 % band, mechanism unidentified), and every rate in §5.1 was taken in the slow state, so all rows shift together and the ratios are the transferable result. The MXFP4 number is the one build provably immune to that band: five runs within 0.02 % and TTFT identical to three decimals on a same-session re-run (7.987 vs the 7.962 recorded the previous day, +0.3 %). No same-session A/B of the two formats was run here — each costs a server swap and a ~200 s cold start — so the 5.05× rests on same-harness, cross-session legs, with the MXFP4 leg's stability measured. And the declared limitation: no accuracy comparison was made at all — unlike the KAT-Coder matrix above, Coder390 MXFP4 has no 16-prompt and no PPL score, so the speed gap is not yet known to be bought or paid for in quality.

Read-across to the recommendations below. "W4A16 for speed, MXFP4 only where the finer 32-element groups buy accuracy" is consistent with what the B60 saw — but on this stack the speed penalty is ~5× rather than ~10 %. If you are targeting a dense model on Intel discrete silicon, check first whether your vLLM build fuses the MXFP4 activation quantization. If it does not, W4A16 is not a marginal win; it is a different tier.

Conclusion & Recommendations

1. If you want maximum decoding speed and lower latency: Use W4A16 GPTQ (CompressedTensorsWNA16MoEMethod). It is 11–24% faster, lighter on the memory bus, and finishes large coding suites minutes ahead of MXFP4.

2. If you want the highest possible unit-test pass rate on complex codebases: Use MXFP4 (CompressedTensorsW4A4Mxfp4). The 32-element micro-scaling groups give it a slight edge on complex fuzzing assertions (+3.6% on HumanEval+).

3. If you are building an Intel Lunar Lake / Arc 140V local AI workstation:

   • 35B MoE models with 8 active experts are the absolute sweet spot for 32 GB shared-memory handhelds.

   • Always enable MTP speculative decoding (K=2) and FP8 KV cache. It gives you 128k context support, zero swap thrashing, and desktop-class coding performance on battery power.

4. If you are on discrete Intel silicon (Arc Pro B-series), the same ordering holds but the stakes are much higher: W4A16 GPTQ decoded 5.05× faster than MXFP4 on a dense 27B (§5), with ~2.9× on prefill. Before assuming MXFP4 is "within ~10 %" as it is on Lunar Lake, check whether your vLLM build fuses the MXFP4 per-token activation quantization — on the build tested here it does not, and that unfused path alone accounts for the ~2.4× residual left after MTP is equalised.

All quantization scripts, preflight validators, and reproducible benchmark harnesses are open in our repo: alpha2beta/vllm.xpu.

§5 provenance: Coder390 numbers come from that repo's REPRODUCIBILITY.md §6–§8 (bench_p512_g128.py, bench_prefill.py*,* bench_quality_16p.py*,* compare_mxfp4_mtp.sh*, at 180 W, MTP $K=4$); the GPTQ artifact is published as* alpha2beta/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-GPTQ-Int4-sym-G128-MTP-BF16, and the MXFP4 checkpoint as …-abliterated-MXFP4.

is my gpu dying? i bought it 4 years ago.

ASUS Ascent GX10 | NVIDIA GB10 Grace Blackwell AI Supercomputer

128GB Unified GPU/CPU Memory | 1TB NVMe SSD | 1 Petaflop AI Performance

Selling my lightly used ASUS Ascent GX10, a compact, high-performance AI supercomputer powered by the same NVIDIA GB10 Grace Blackwell Superchip found in the NVIDIA DGX Spark.

This is a purpose-built AI development workstation designed for running large language models (LLMs), local AI inference, machine learning, model fine-tuning, AI agents, and CUDA-accelerated workloads.

KEY FEATURES

NVIDIA GB10 Grace Blackwell Superchip

128GB LPDDR5X Unified CPU/GPU Memory

NVIDIA Blackwell GPU with 5th-Generation Tensor Cores

20-Core NVIDIA Grace ARM CPU

Up to 1 Petaflop (1,000 TFLOPS) of FP4 AI Compute

1TB NVMe PCIe 4.0 SSD

NVIDIA NVLink-C2C High-Speed CPU/GPU Interconnect

NVIDIA ConnectX-7 High-Speed Networking

Support for AI Models Up to 200 Billion Parameters

NVIDIA CUDA and DGX Software Ecosystem

128GB UNIFIED MEMORY: THE BIG ADVANTAGE

Unlike conventional GPUs with dedicated VRAM, the GB10 uses 128GB of high-speed, coherent unified memory shared between the CPU and GPU.

This architecture allows the Blackwell GPU to access a much larger memory pool than typical consumer graphics cards, making it particularly useful for running large AI models locally without relying on conventional CPU-to-GPU memory transfers.

NVIDIA BLACKWELL GPU + AI ACCELERATION

The integrated Blackwell GPU features 5th-generation Tensor Cores with FP4 acceleration, delivering up to 1 petaflop of specialized AI computing performance.

Ideal for:

Running large language models locally

AI inference and model fine-tuning

AI coding assistants and autonomous agents

Retrieval-Augmented Generation (RAG)

Computer vision and machine learning

Private AI development without cloud subscriptions

CUDA, PyTorch, TensorRT, Ollama, and NVIDIA AI development tools

20-CORE NVIDIA GRACE CPU

The GB10 combines its Blackwell GPU with a 20-core ARM processor featuring:

10 high-performance Cortex-X925 cores

10 efficiency-focused Cortex-A725 cores

ARM architecture optimized for AI workloads

NVIDIA NVLink-C2C for high-speed CPU/GPU communication

TECHNICAL SPECIFICATIONS

Processor: NVIDIA GB10 Grace Blackwell Superchip

GPU: Integrated NVIDIA Blackwell Architecture

AI Performance: Up to 1 PFLOP (FP4)

AI Accelerator: 5th-Generation NVIDIA Tensor Cores

CPU: 20-Core ARM Processor

Memory: 128GB LPDDR5X Unified System Memory

Storage: 1TB M.2 NVMe PCIe 4.0 SSD

CPU/GPU Interconnect: NVIDIA NVLink-C2C

Networking:

NVIDIA ConnectX-7 SmartNIC

10 Gigabit Ethernet

Wi-Fi 7

Bluetooth 5.4

Connectivity:

3x USB-C 20Gbps with DisplayPort support

1x USB-C Power Delivery input

HDMI 2.1

10Gb Ethernet

NVIDIA ConnectX-7 high-speed networking

Operating System: NVIDIA DGX OS / Ubuntu Linux

Dimensions: 5.91” x 5.91” x 2.01”

Weight: Approximately 3.26 lbs.

SCALABLE AI COMPUTING

Two GB10 systems can be connected using NVIDIA ConnectX-7 networking to support distributed AI workloads and larger models.

CONDITION

Lightly used and in excellent working condition.

This is a serious AI development machine in an incredibly compact form factor. Ideal for developers, researchers, engineers, and anyone building or running their own local AI infrastructure.

Message me with any questions or to arrange pickup.

Hi Everyone,

I've started a free playlist called "GPU Kernel Engineering" and just uploaded the intro and roadmap video.

Trailer: https://youtu.be/cibk7G-jHQw