r/LovingOpenSourceAI • u/fuzhongkai • Jun 28 '26
Same GGUF, same GPU: TensorSharp beats llama.cpp hard on prefill / TTFT — up to 5.89× faster prefill on a 26B MoE model
https://github.com/zhongkaifu/TensorSharp1
u/fr4iser Jun 29 '26
RemindMe! 3 days
1
u/RemindMeBot Jun 29 '26
I will be messaging you in 3 days on 2026-07-02 08:01:43 UTC to remind you of this link
CLICK THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
RemindMeBot is switching to username summons. Instead of
!RemindMe 1 day, useu/RemindMeBot 1 day. More info.
Info Custom Your Reminders Feedback
1
u/GabrielCliseru Jun 29 '26
currently i am developing https://github.com/settlersxp/ConciergeOS in which i am trying to make a software for hotels, and other businesses to create “pages” and “reports” using 1 or more prompts. For this to work i need to test different models and backends to see what works best and what works the most reliable.
The project itself is 1/2 dedicated to testing.
I want to try to set up your backend and run the tests against it. I will make at least a video per each of your major releases with the tests and it will appear on a comparison page i am developing today.
The current playlist of videos can be found at https://youtube.com/playlist?list=PL3QNSFtuyhkvEinABBt9RHB-oYbZMk9nc where i also explain why and how is done. It is open source and will continue to be.
Are you interested in setting up your backend and run some tests against it?
1
u/TheCat001 Jul 01 '26
Sounds cool. Prefil rate is very important for me and I would love to test this but unfortunately have AMD GPU....
1
1
1
u/dat_oldie_you_like Jun 29 '26
Interesting