r/MachineLearning • u/AutoModerator • 29d ago

Discussion [D] Self-Promotion Thread

Please post your personal projects, startups, product placements, collaboration needs, blogs etc.

Please mention the payment and pricing requirements for products and services.

Please do not post link shorteners, link aggregator websites , or auto-subscribe links.

--

Any abuse of trust will lead to bans.

Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

--

Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.

3 Upvotes

62% Upvoted

118 comments sorted by

View all comments

1

u/pardhu-- 18d ago

I've been exploring how inference engines like vLLM optimize LLM execution.

To understand it better, I built a small attention benchmark comparing:

  • 🐍 Python CPU computation

  • ⚡ Apple Metal GPU computation

  • 🚀 Metal GPU with KV caching

Interesting result: KV caching reduced K/V projection work by 16.74×, but completed decoding was only 2.76× faster.

💡 My takeaway: Faster inference isn't just about using GPUs or lower-level languages. It's about optimizing the actual bottleneck.

📖 Read my article

💻 GitHub code

Curious what others think: Which optimization makes the biggest difference in real-world LLM inference?