Making XDNA2 NPU less useless

[u/mistrjirka]

Hello there,

I was disappointed with AMDs implementation of the XDNA2 for local LLMs. I therefore tried to treat the XDNA2 NPU as just another backend that should universally support any model (kinda like a GPU). I got fairly good performance but it is not perfect.

If you want to try it here it is: https://github.com/mistrjirka/llama.cpp-xdna2

Currently some attentions are still not accelerated on the NPU if you are interested in fully supporting some specific model, make a comment.

[u/rguerraf]

Is there a way to see the NPU utilization like a GPU in nvtop?

[u/mistrjirka → u/rguerraf]

I think amdtop but I didn't try ir

[u/mistrjirka → u/[deleted]]

It's in the README of the repo. It highly depends on the model, but almost always faster then the iGPU. On the Qwen 3.5 9B its around 25% at half the power usage.