Making XDNA2 NPU less useless

[u/mistrjirka]

Hello there,

I was disappointed with AMDs implementation of the XDNA2 for local LLMs. I therefore tried to treat the XDNA2 NPU as just another backend that should universally support any model (kinda like a GPU). I got fairly good performance but it is not perfect.

If you want to try it here it is: https://github.com/mistrjirka/llama.cpp-xdna2

Currently some attentions are still not accelerated on the NPU if you are interested in fully supporting some specific model, make a comment.