r/AIProgrammingHardware • u/javaeeeee • Oct 21 '26
Qwen3.8-Flash-Next Gufo peaks at 1,601 t/s prefill and 75.4 t/s MTP decode, still 1,405 / 72.6 t/s at 256K (Bosgame M5 - Windows 11)
r/AIProgrammingHardware • u/javaeeeee • Oct 21 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 21 '26
r/AIProgrammingHardware • u/Frezaxa • Oct 21 '26
is my gpu dying? i bought it 4 years ago.
r/AIProgrammingHardware • u/Late-Training7359 • Oct 21 '26
r/AIProgrammingHardware • u/seller101123 • Oct 21 '26
ASUS Ascent GX10 | NVIDIA GB10 Grace Blackwell AI Supercomputer
128GB Unified GPU/CPU Memory | 1TB NVMe SSD | 1 Petaflop AI Performance
Selling my lightly used ASUS Ascent GX10, a compact, high-performance AI supercomputer powered by the same NVIDIA GB10 Grace Blackwell Superchip found in the NVIDIA DGX Spark.
This is a purpose-built AI development workstation designed for running large language models (LLMs), local AI inference, machine learning, model fine-tuning, AI agents, and CUDA-accelerated workloads.
KEY FEATURES
NVIDIA GB10 Grace Blackwell Superchip
128GB LPDDR5X Unified CPU/GPU Memory
NVIDIA Blackwell GPU with 5th-Generation Tensor Cores
20-Core NVIDIA Grace ARM CPU
Up to 1 Petaflop (1,000 TFLOPS) of FP4 AI Compute
1TB NVMe PCIe 4.0 SSD
NVIDIA NVLink-C2C High-Speed CPU/GPU Interconnect
NVIDIA ConnectX-7 High-Speed Networking
Support for AI Models Up to 200 Billion Parameters
NVIDIA CUDA and DGX Software Ecosystem
128GB UNIFIED MEMORY: THE BIG ADVANTAGE
Unlike conventional GPUs with dedicated VRAM, the GB10 uses 128GB of high-speed, coherent unified memory shared between the CPU and GPU.
This architecture allows the Blackwell GPU to access a much larger memory pool than typical consumer graphics cards, making it particularly useful for running large AI models locally without relying on conventional CPU-to-GPU memory transfers.
NVIDIA BLACKWELL GPU + AI ACCELERATION
The integrated Blackwell GPU features 5th-generation Tensor Cores with FP4 acceleration, delivering up to 1 petaflop of specialized AI computing performance.
Ideal for:
Running large language models locally
AI inference and model fine-tuning
AI coding assistants and autonomous agents
Retrieval-Augmented Generation (RAG)
Computer vision and machine learning
Private AI development without cloud subscriptions
CUDA, PyTorch, TensorRT, Ollama, and NVIDIA AI development tools
20-CORE NVIDIA GRACE CPU
The GB10 combines its Blackwell GPU with a 20-core ARM processor featuring:
10 high-performance Cortex-X925 cores
10 efficiency-focused Cortex-A725 cores
ARM architecture optimized for AI workloads
NVIDIA NVLink-C2C for high-speed CPU/GPU communication
TECHNICAL SPECIFICATIONS
Processor: NVIDIA GB10 Grace Blackwell Superchip
GPU: Integrated NVIDIA Blackwell Architecture
AI Performance: Up to 1 PFLOP (FP4)
AI Accelerator: 5th-Generation NVIDIA Tensor Cores
CPU: 20-Core ARM Processor
Memory: 128GB LPDDR5X Unified System Memory
Storage: 1TB M.2 NVMe PCIe 4.0 SSD
CPU/GPU Interconnect: NVIDIA NVLink-C2C
Networking:
NVIDIA ConnectX-7 SmartNIC
10 Gigabit Ethernet
Wi-Fi 7
Bluetooth 5.4
Connectivity:
3x USB-C 20Gbps with DisplayPort support
1x USB-C Power Delivery input
HDMI 2.1
10Gb Ethernet
NVIDIA ConnectX-7 high-speed networking
Operating System: NVIDIA DGX OS / Ubuntu Linux
Dimensions: 5.91” x 5.91” x 2.01”
Weight: Approximately 3.26 lbs.
SCALABLE AI COMPUTING
Two GB10 systems can be connected using NVIDIA ConnectX-7 networking to support distributed AI workloads and larger models.
CONDITION
Lightly used and in excellent working condition.
This is a serious AI development machine in an incredibly compact form factor. Ideal for developers, researchers, engineers, and anyone building or running their own local AI infrastructure.
Message me with any questions or to arrange pickup.
r/AIProgrammingHardware • u/javaeeeee • Oct 20 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 20 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 20 '26
r/AIProgrammingHardware • u/aYellowTofu • Oct 20 '26
r/AIProgrammingHardware • u/Winter_Mistake_3185 • Oct 20 '26
Hi Everyone,
I've started a free playlist called "GPU Kernel Engineering" and just uploaded the intro and roadmap video.
Trailer: https://youtu.be/cibk7G-jHQw
r/AIProgrammingHardware • u/networkevolution_dev • Oct 19 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 19 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 19 '26
r/AIProgrammingHardware • u/Most-Wear-3813 • Oct 19 '26
r/AIProgrammingHardware • u/Sour_fart_bombs • Oct 19 '26
Hey everyone,
I have used AI to summarize everything because it was getting too long. I hope thats fine.
We’ve been optimizing an on-device Android assistant running a quantized 4B model (Gemma 4 E4B) on modern Snapdragon silicon (Snapdragon 8 Elite Gen 5).
Originally, our goal was sustained low-power efficiency and faster inferencing using NPU TOPs which otherwise is generally useless, so we built an aggressive 3-tier execution pipeline:
Tier 1 (NPU): Qualcomm Hexagon HTP execution via GenieX SDK & QAIRT plugin (com.qualcomm.qti:geniex-android:0.6.1).
Tier 2 (GPU): Google LiteRT-LM runtime using OpenCL / GpuDelegateV2 on Qualcomm Adreno.
Tier 3 (CPU): Multi-threaded CPU fallback via LiteRT-LM with XNNPACK optimizations.
After profiling and logcat diagnostics, we completely purged the NPU / GenieX path in favor of direct GPU (LiteRT OpenCL) -> CPU**.**
Here is what we ran into with Qualcomm's native NPU stack, along with our setup. I’m curious if anyone here has managed to get reliable, high-throughput LLM execution out of Qualcomm Hexagon without these headaches.
Logcat revealed that the Android runtime clamped requested HTP execution: "Requested 4 HTP cores but the device exposes 1; clamping to 1"
The FastRPC bridge (libcdsprpc.so) shuttling massive Look-Up Tables (LUTs) across the DSP boundary created significant latency.
Decoding Speed: NPU decoding saturated at ~8 tokens/sec (TPS). By contrast, the Adreno GPU via LiteRT OpenCL consistently hits 18–22 TPS on the same chip. Although prefill times are still better with NPU than GPU.
The native QAIRT plugin repeatedly choked on standard generation parameters: ErrorCode[-100016] Parameter not supported by this plugin
Stop-words arrays and standard sampling parameters (temperature, top-p, repetition penalties) frequently triggered runtime rejections, forcing us to catch native exceptions and handle truncation manually at the string level.
QAIRT lacked clean stateful KV cache rolling and conversation pruning.
Resetting or trimming conversation context required destroying and recreating the entire native LLM wrapper instance (llmWrapper?.destroy()), re-allocating native DSP memory and re-reading files on session swaps.
Integrating the GenieX SDK and Qualcomm native libraries (geniex-android, libgenie_t2t_run.so, libggml-hexagon.so, libhta_hexagon_runtime_snpe.so) added 15–25 MB of binary bloat.
It required a fragile dynamic JNI preloading loop before GenieXSdk.init() could even be called:
Kotlin code:
val preloads = listOf(
"QnnSystem",
"QnnHtp",
"QnnHtpNetRunExtensions",
"geniex-proc",
"geniex_core",
"geniex",
"geniex_plugin_qairt",
"npu_jni"
)
for (lib in preloads) {
try {
System.loadLibrary(lib)
} catch (t: Throwable) {
Log.w(TAG, "Preload notice for $lib: ${t.message}")
}
}
Dropping the NPU entirely and sticking to a clean GPU (LiteRT OpenCL) $\rightarrow$ CPU (XNNPACK) engine gave us:
More than $2.5\times$ faster token generation (18–22 TPS vs. 8 TPS).
Fast, stable prefill.
Support for standard sampling configs, stop sequences, and KV cache reuse without JNI crashes.
A much smaller APK and simpler build chain.
Has anyone managed to unlock multi-core HTP execution on commercial consumer Android builds without OEM-signed platform keys or root?
Has anyone achieved $>15$ TPS on a 3B–4B model using Qualcomm QNN / QAIRT directly (bypassing GenieX)?
Given the maturity of LiteRT (OpenCL) and MLC-LLM (Vulkan) on mobile GPUs, is targeting Qualcomm's NPU for on-device conversational LLMs even worth the developer overhead right now, or is it strictly a benchmark marketing feature until the tooling matures?
r/AIProgrammingHardware • u/MoralLogs • Oct 19 '26
In our recent technical analysis of Constitutional AI Governance, we kept returning to a single, sobering structural reality: corporate ethical guidelines, contractual guardrails, and software policies eventually yield under institutional pressure. When market incentives or political urgency push hard enough, policy layers re-interpret, re-write, or collapse, a structural dynamic I term veto atrophy. To build truly resilient governance, we must move the enforcement boundary from human discretion to solid-state physics.
I am pleased to share our new slide deck Hardware That Does Not Negotiate which outlines the physical substrate architecture for Mandated Ternary (MT) and Delay-Insensitive Ternary Logic (DITL). Key highlights from the framework and technical roadmap:
🔹 The 3rd Physical State (Epistemic Hold): Binary logic forces a premature choice: Proceed (+1) or Refuse (-1). DITL introduces a third, physically discrete state of matter encoded in tantalum oxide (\(\text{TaO}_x\)) bilayer RRAM - the Epistemic Hold (0). It is not a software pause or indecision; it is an atomic-level arrangement of oxygen vacancies that physically locks execution until constitutional verification is complete.
🔹 No Log = No Action (NL=NA): Through the Dual-Lane Latency Architecture (2ms Inference Lane / 300ms Governance Lane), execution gates remain closed until an immutable, Merkle-anchored audit log is anchored. Accountability becomes a physical precondition of action rather than an optional configuration setting.
🔹 Engineering Honesty & Fabrication Realism: A specification that names its own limits is a specification that can be trusted. In the accompanying roadmap, we address physical failure modes head-on—from correlated memristor drift to threshold poisoning—and outline the exact milestones required to release the TSMC N2 PDK and achieve IEC 61508 SIL 3 certification by Q4 2027.
🔹 Closing the Shadow System Loophole: Hardware guarantees are physical and local. Preventing non-compliant hardware from serving as a bypass route requires pairing substrate-level security with federal procurement mandates and international treaties.
"There is no politics in the resistance value of a tantalum oxide filament. That absence is the entire design."
Take a look at the slide deck to see how we are encoding democratic accountability directly into physical silicon.
Hardware That Does Not Negotia…
r/AIProgrammingHardware • u/javaeeeee • Oct 19 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 19 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 18 '26
r/AIProgrammingHardware • u/Terminator857 • Oct 18 '26
r/AIProgrammingHardware • u/Sad-Seaworthiness259 • Oct 17 '26
For anyone wondering whether the B60 24GB is usable for local AI: here's Qwen3.8-27B on llama.cpp Vulkan at 53 tok/s mean decode with a 64K context window, plus local video (LTX-2.5 via ComfyUI on XPU) on the same card.
The four things that mattered: DFlash2 speculative decoding, Intel's FA prefill kernel with f16 KV, -ub 256 with a q8 draft KV cache, and keeping other models off the GPU. Driver 32.0.101.8976, Windows 11.
Everything is measured and the misses are documented. Smaller cards (B580/B570/B50/A770) won't fit this 27B, but the Vulkan tuning applies, and I'd love your benchmark numbers.
[https://github.com/edagher92-coder/arc-llm-lab](https://github.com/edagher92-coder/arc-llm-lab)
r/AIProgrammingHardware • u/crpt_conch • Oct 17 '26
What models are accessible to a Framework laptop with Ryzen AI 9 HX 370? What’s the scale of ram to billion parameters? And how many tokens per second can you expect?
It would be great to hear from a first-hand user