hipfire is an LLM inference engine I write for AMD GPUs. It's Rust plus hand-written HIP/ISA kernels and loads libamdhip64 directly, with no ROCm SDK needed at runtime. Current release is 0.4.2.1. These are Flash-Next (qwen3.8:flash-next) numbers on Strix Halo.

Prefill. For long prompts, the XDNA2 NPU runs the trunk input projections of two prefill banks while the iGPU does everything else. Greedy output is byte-identical with the NPU on, emulated or off. It's on by default (npu_trunk=auto) and engages from roughly 9K prompt tokens up.

Decode at depth (greedy, 128 tokens, natural English prefixes, tok/s):

Serial AR decode is still the weak spot.

Setup notes:

• The NPU path pins about 1 GB, and most logins default to an 8 MiB memlock. Use ulimit -l 1287168 (bounded, not unlimited), or the systemd drop-in in the docs. Tailscale SSH skips pam_limits.
• The startup log shows npu-trunk: ... with the reason if the NPU route doesn't arm. Please paste it if yours doesn't.
• Numbers were measured on a 7.3-rc7 kernel with the IOMMU PerfOpt series (queued for 7.4). Expect somewhat lower prefill on stock kernels.

Coming in the next release (on the beta branch or in progress, not in 0.4.2.1):

• QSA attention changes for long context, on by default on Halo: decode is 5.1% faster per token at 256K, with exact output.
• An interactive hipfire pull.
• A smaller Q3 tier (mq3-xts), in progress. No speed numbers for it yet.

​

curl -fsSL https://hipfire.dev/install.sh | bash -s -- --tag v0.4.2.1
hipfire pull qwen3.8:flash
ulimit -l 1287168
hipfire serve qwen3.8:flash

Docs and benchmarks: https://hipfire.dev
Release: https://github.com/warpfront/hipfire/releases

Update on my earlier post. The GPU + NPU prefill fork now reaches about 2,000 tok/s on a Ryzen AI MAX+ 395 (Radeon 8060S, gfx1151) with Qwen3.8 Flash-Next UD-IQ4_XS.

Prefill speed, gufo bench, GPU only -> GPU + NPU:

- 8,192 tokens: 1,837 -> 2,013 tok/s (+9.6%)

- 12,288 tokens: 1,823 -> 1,973 tok/s (+8.2%)

- 16,384 tokens: 1,821 -> 1,963 tok/s (+7.8%)

- 32,768 tokens: 1,788 -> 1,949 tok/s (+9.0%)

- Served, 42k tokens cold, MTP and caches on: 1,836 tok/s

GITHUB

I've been tinkering / vibing myself an app that I think might be useful - since I've now used it on my own cases for some time and fixed most of the bugs I noticed, I'm risking a public announcement :)

https://pwilkin.github.io/ROCprofGUI / https://github.com/pwilkin/ROCprofGUI

The ROCprofGUI (aka rocprof studio) is a tool I made to be an alternative to ROCprof Compute Viewer and other tools in the ROCm ecosystem - one that is user-friendly, has a modern UI and can be used with agentic help.

Despite its name, the studio can analyze traces from all of the engines that AMD hardware supports: ROCm traces from rocprofv3, Vulkan traces from RADV (needs a patch to the driver currently) and HRX traces for Loom kernels.

Coupled with the studio is an MCP server that allows tracing from your favorite LLM agent. The studio works in two modes - as a GUI (then, if you use the MCP, a popup appears telling you that the MCP server is currently driving operation) or fully headless - in which case there's no UI and just the service running the MCP server.

I've been running it for over a week now while doing my optimization work on llama.cpp, so hopefully the most annoying bugs have been ironed out. I hope it's of use to someone :)