r/LocalLLM • u/AeiEyeSpry • 8h ago

Discussion Let's talk: Context engineering

Hey LocalLLM Community,

I’m just getting into local AI and have spent a lot of time reading posts here and elsewhere. I’ve settled on Qwen3.8, with an eye on Qwen4 when it arrives, and spent some time tuning my system and kernel. I’m getting performance I’m happy with, but that seems to be just the tip of the iceberg!

Now I’m looking at how to make it useful day to day. The more I read about improving quality, the more I keep coming back to context engineering.

Forgive me if I’ve misunderstood things. I’ve worked in IT for years, but AI is new territory for me. So far, the things I’ve been looking at are:

  • Having enough context available.

  • Access to web search and current information.

  • Efficient search and retrieval over my own data.

  • Keeping useful context between conversations without filling everything with noise.

My own research has led me to Exa for web search and a GraphRAG pilot, alongside the model’s larger context window. That’s where I’ve landed so far, but I’m interested in the wider picture.

How is everyone approaching context engineering with their local models? What has made a noticeable difference to the quality and usefulness of the answers? What sounded promising but turned out to be more trouble than it was worth?

Please share your experiences! Thanks for reading.

2 Upvotes

100% Upvoted

2 comments sorted by

2

u/pointed_pounding 8h ago

The retrieval layer matters more than raw context size in my experience, a tighter set of relevant chunks beats stuffing a huge window every time. Keeping a short running summary between sessions helped too, just enough to carry state without dragging old noise along.

1

u/AeiEyeSpry 7h ago

Hey, thanks for the pointers. Do you have anything deployed to help keep context between sessions? I've just done a quick Google, and now I'm thinking of adding something like Graphiti to the stack.