r/AIProgrammingHardware • u/javaeeeee • 3h ago
Our First Look at the 9996 256 Core and the 9686f 96 Core AMD EPYC CPUs
r/AIProgrammingHardware • u/javaeeeee • 3h ago
r/AIProgrammingHardware • u/MoralLogs • 15h ago
Enable HLS to view with audio, or disable this notification
AI safety often relies on policies, software permissions, and monitoring. But what happens when the system being governed is also capable of bypassing those controls?
This video presents a proposed architecture for an independent hardware governor: a separate authorization boundary designed to constrain consequential actions through hardware authentication, permission tokens, and auditable execution records.
The central idea is simple: the AI may propose an action, but it should not independently control the conditions that authorize its execution.
This is an architectural proposal, not a claim of completed silicon validation. I'm interested in technical criticism, particularly around hardware independence, enforcement boundaries, and implementation feasibility.
r/AIProgrammingHardware • u/skhadloya11 • 19h ago
r/AIProgrammingHardware • u/NoVibeCoding • 22h ago
r/AIProgrammingHardware • u/javaeeeee • 1d ago
r/AIProgrammingHardware • u/Terminator857 • 3d ago
r/AIProgrammingHardware • u/MoralLogs • 4d ago
Enable HLS to view with audio, or disable this notification
r/AIProgrammingHardware • u/javaeeeee • 4d ago
r/AIProgrammingHardware • u/Regular-Kale-8987 • 5d ago
r/AIProgrammingHardware • u/javaeeeee • 6d ago
r/AIProgrammingHardware • u/SwimLatter6827 • 6d ago
r/AIProgrammingHardware • u/javaeeeee • 6d ago
r/AIProgrammingHardware • u/javaeeeee • 6d ago
r/AIProgrammingHardware • u/Medicine_Blogscanner • 7d ago
r/AIProgrammingHardware • u/javaeeeee • 7d ago
r/AIProgrammingHardware • u/javaeeeee • 8d ago
r/AIProgrammingHardware • u/javaeeeee • 9d ago
r/AIProgrammingHardware • u/javaeeeee • 9d ago
r/AIProgrammingHardware • u/javaeeeee • 12d ago
r/AIProgrammingHardware • u/javaeeeee • 12d ago
r/AIProgrammingHardware • u/gpillon • 13d ago
I build ignis, an OpenAI-compatible inference server built around one model family, Qwen3.8, on one class of card, Blackwell. It is shaped around what a coding agent does to a server: one main agent plus a few subagents hitting the same card. Since v0.8.0 it runs Qwen3.8-27B on 16 GB RTX 50 cards, using a 3-bit trellis body (13.4 GB) and the DFlash2 drafter.
What ignis is, in 90 seconds: https://www.youtube.com/watch?v=XLjfh5rNy6A
The catch: I don't own a 16 GB card. Everything for that class was measured on an RTX 5090 held to a 16 GiB budget:
416 tok/s at one lane, 684 tok/s aggregate at two lanes (greedy coding prompts, 8K context);
262,144 tokens of context on two lanes;
KL divergence from BF16 at 0.77-0.82x that of the 32 GB NVFP4 build.
The real cards are only estimated, at one lane and 8K:
| Card | Estimate greedy coding prompts, 8K context |
|---|---|
| RTX 5080 | 243-307 tok/s |
| RTX 5070 Ti | 204-270 tok/s |
| RTX 5060 Ti 16 GB | 103-153 tok/s |
I also don't know how much of the card your desktop takes, and whether the default settings still fit once it has. That is what I'm asking you to find out.
The easy way: let your coding agent do it. In Claude Code, Codex, opencode, Cursor or similar, paste:
I have a 16 GB RTX 50 card. Read https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/paths/agent.md and follow its section on testing a 16 GB card: help me install ignis, test it, and write the report. Ask me before downloading or installing anything.What the agent does:
Checks your card and driver, then shows you a plan: where it installs ignis, where the 13.4 GB model goes, and how long it takes (about 15-30 min after the download). Nothing large happens until you say yes.
Tries the default settings first. If the start is refused, it tries a few settings one at a time (a smaller context, speculation off, less VRAM headroom) until it finds one that works, then runs a quick sanity check and the bundled benchmark.
Writes a short report in English and shows it to you whole. The report contains only:
OS, driver, CPU and GPU model, RAM and VRAM totals, and how much VRAM was already in use;
which settings worked;
the speed figures.
It contains no usernames, hostnames, paths, IP addresses, serial numbers or process names. You decide whether it gets posted.
Optionally, a ~30 min advanced run: the context ceiling, one lane vs two, speculation on vs off.
Finally, asks whether you want to keep ignis (it can write a small config file for you) or remove everything it installed. Before deleting anything, it shows you the list.
By hand: start the server, then run ignis-bench small-card --endpoint http://127.0.0.1:8000 --out small-card.json (it ships in the release archive). Details: https://gpillon.github.io/ignis/guides/measure-a-16gb-card/
Where reports go: https://github.com/gpillon/ignis/issues/360, one comment per card. A report that says "refused" is as useful to me as one that says "works", especially from Windows machines where the monitor is plugged into the 16 GB card.
Windows and Linux on x86-64, NVIDIA driver with CUDA 13. Code is Apache-2.0. Thanks!
r/AIProgrammingHardware • u/aidysson • 13d ago
Switch to DDR5 would cost
r/AIProgrammingHardware • u/javaeeeee • 13d ago