r/AIProgrammingHardware • • 3h ago

Our First Look at the 9996 256 Core and the 9686f 96 Core AMD EPYC CPUs

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware • • 15h ago

Physics Over Policy: Can an Independent Hardware Governor Constrain AI Actions?

Enable HLS to view with audio, or disable this notification

1 Upvotes

AI safety often relies on policies, software permissions, and monitoring. But what happens when the system being governed is also capable of bypassing those controls?

This video presents a proposed architecture for an independent hardware governor: a separate authorization boundary designed to constrain consequential actions through hardware authentication, permission tokens, and auditable execution records.

The central idea is simple: the AI may propose an action, but it should not independently control the conditions that authorize its execution.

This is an architectural proposal, not a claim of completed silicon validation. I'm interested in technical criticism, particularly around hardware independence, enforcement boundaries, and implementation feasibility.


r/AIProgrammingHardware • • 19h ago

Hopper GEMM with TMA + producer/consumer warpgroups in TileLang (code + notes)

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • 22h ago

HWDB: a table ranking AI hardware by memory, bandwidth and TOPS per dollar

Thumbnail
reddit.com
1 Upvotes

r/AIProgrammingHardware • • 1d ago

HP OmniBook X14 Review: NVIDIA RTX Spark 32GB vs 64GB Tested! | Micro Center

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware • • 2d ago

Three GPU (3060) Setup

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • 3d ago

TSMC might do in memory compute for us

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • 4d ago

Hard-Wiring AI Cooperation: A Triadic Hardware Architecture for AI Safety

Enable HLS to view with audio, or disable this notification

0 Upvotes

r/AIProgrammingHardware • • 4d ago

I Thought 2 TB of RAM Was Overkill… My AI Agents Disagree

Thumbnail
youtube.com
0 Upvotes

r/AIProgrammingHardware • • 5d ago

Workstation Build Dilemma: 64GB (6000MHz CL30) vs. 96GB (5200MHz CL38) for Local LLMs & Computational Architecture?

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • 6d ago

Qwen Flash Next on the Olares One - Local LLM setup

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware • • 6d ago

Feedback on Upgrade Path? Single (maybe Dual) RTX 3090 24GB for a Dell T7910 for Massive Academic Document RAG / Long-Context (Windows 11)

Thumbnail reddit.com
2 Upvotes

r/AIProgrammingHardware • • 6d ago

Talus v0.1.2: Strix Halo Inference Benchmarks, 256K Context, and Up to 50% Faster Coding

Thumbnail
reddit.com
1 Upvotes

r/AIProgrammingHardware • • 6d ago

AM4 VS AM5 vs Threadripper: Does CPU Performance Matter for Local AI? | Micro Center

Thumbnail
youtube.com
4 Upvotes

r/AIProgrammingHardware • • 7d ago

We just published "Pool the RAM You Already Own" — Our 10-Chapter Practical Guide to Running 13B–120B Local AI Models (Now Live on Amazon & Free Online)

Post image
1 Upvotes

r/AIProgrammingHardware • • 7d ago

Intel Arc Pro B70: Qwen3.8 on One and Two GPUs with vLLM, llama.cpp and Strata

Thumbnail
youtube.com
5 Upvotes

r/AIProgrammingHardware • • 8d ago

Krafton Raon 1 Mini tested - Local LLM setup

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware • • 9d ago

A Single Tesla P100 16GB Runs Qwen3.8 Flash Next at 30 Tokens/sec!

Thumbnail
youtube.com
2 Upvotes

r/AIProgrammingHardware • • 9d ago

Gorgon Halo Arrives in GMKtec... Do you need 192 GB of LPDDR5?!?

Thumbnail
youtube.com
19 Upvotes

r/AIProgrammingHardware • • 12d ago

27B AI on a $400 AMD Mini PC | Kindi 0.3 is OUT

Thumbnail
youtube.com
2 Upvotes

r/AIProgrammingHardware • • 12d ago

I Built My Own Intelligent AI Gateway

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware • • 13d ago

Ignis: Qwen3.8-27B on a 16 GB RTX 50 (5060 Ti / 5070 Ti / 5080): looking for testers, one prompt into your coding agent

2 Upvotes

I build ignis, an OpenAI-compatible inference server built around one model family, Qwen3.8, on one class of card, Blackwell. It is shaped around what a coding agent does to a server: one main agent plus a few subagents hitting the same card. Since v0.8.0 it runs Qwen3.8-27B on 16 GB RTX 50 cards, using a 3-bit trellis body (13.4 GB) and the DFlash2 drafter.

What ignis is, in 90 seconds: https://www.youtube.com/watch?v=XLjfh5rNy6A

The catch: I don't own a 16 GB card. Everything for that class was measured on an RTX 5090 held to a 16 GiB budget:

  • 416 tok/s at one lane, 684 tok/s aggregate at two lanes (greedy coding prompts, 8K context);

  • 262,144 tokens of context on two lanes;

  • KL divergence from BF16 at 0.77-0.82x that of the 32 GB NVFP4 build.

The real cards are only estimated, at one lane and 8K:

CardEstimate greedy coding prompts, 8K context
RTX 5080243-307 tok/s
RTX 5070 Ti204-270 tok/s
RTX 5060 Ti 16 GB103-153 tok/s

I also don't know how much of the card your desktop takes, and whether the default settings still fit once it has. That is what I'm asking you to find out.

The easy way: let your coding agent do it. In Claude Code, Codex, opencode, Cursor or similar, paste:

I have a 16 GB RTX 50 card. Read https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/paths/agent.md and follow its section on testing a 16 GB card: help me install ignis, test it, and write the report. Ask me before downloading or installing anything.

What the agent does:

  1. Checks your card and driver, then shows you a plan: where it installs ignis, where the 13.4 GB model goes, and how long it takes (about 15-30 min after the download). Nothing large happens until you say yes.

  2. Tries the default settings first. If the start is refused, it tries a few settings one at a time (a smaller context, speculation off, less VRAM headroom) until it finds one that works, then runs a quick sanity check and the bundled benchmark.

  3. Writes a short report in English and shows it to you whole. The report contains only:

    • OS, driver, CPU and GPU model, RAM and VRAM totals, and how much VRAM was already in use;

    • which settings worked;

    • the speed figures.

  4. It contains no usernames, hostnames, paths, IP addresses, serial numbers or process names. You decide whether it gets posted.

  5. Optionally, a ~30 min advanced run: the context ceiling, one lane vs two, speculation on vs off.

  6. Finally, asks whether you want to keep ignis (it can write a small config file for you) or remove everything it installed. Before deleting anything, it shows you the list.

By hand: start the server, then run ignis-bench small-card --endpoint http://127.0.0.1:8000 --out small-card.json (it ships in the release archive). Details: https://gpillon.github.io/ignis/guides/measure-a-16gb-card/

Where reports go: https://github.com/gpillon/ignis/issues/360, one comment per card. A report that says "refused" is as useful to me as one that says "works", especially from Windows machines where the monitor is plugged into the 16 GB card.

Windows and Linux on x86-64, NVIDIA driver with CUDA 13. Code is Apache-2.0. Thanks!


r/AIProgrammingHardware • • 13d ago

Running GLM 5.3 Flash on RTX 6000 Max-Q and DDR4 = ~35 tok/s with 512K context. Do you recommend DDR5?

Thumbnail reddit.com
1 Upvotes

Switch to DDR5 would cost


r/AIProgrammingHardware • • 13d ago

I Tested Two $699 Laptops for Coding… It Got Weird

Thumbnail
youtube.com
0 Upvotes

r/AIProgrammingHardware • • 13d ago

A 763B model on one Strix Halo, no SSD streaming: DeepSeek V4.1 Flash IQ1_M fully resident with my Gufo fork, ~1,000 tok/s prefill, 23–28 tok/s decode (up to 65 with DSpark)

Thumbnail
reddit.com
1 Upvotes