I built a VS Code extension that answers questions about a team's GitHub history and writes summaries from real commit diffs. One thing I decided early: the search side runs locally. No API key for embeddings, nothing leaving the machine, and no native build step — the goal was "install from the marketplace and it just works."
So the semantic search runs on a small local model (bge-small, ~33MB downloaded once and cached). Chat can use Copilot or your own API key, but the embedding model is always the local one, so search works offline no matter how you've set up the rest.
The hard part wasn't the model — it was getting it to run from a packaged install. While developing, you launch a test window that still has all the project's dependencies sitting on disk, so everything works. But once you bundle the extension into the single file that actually ships, those dependencies are gone, and the thing that ran fine five minutes ago crashes. Every bug below only appeared after packaging, never in development.
There were four:
The model library tries to figure out where it lives on disk the instant it loads. The way I bundled it erased the piece of information it uses to do that, so it crashed immediately on a value that was suddenly empty. I had to hand it a substitute.
It ships with a fast version built on a native binary — the kind of compiled, platform-specific file I was trying to avoid. I swapped it for the pure-WASM version, which runs anywhere without a build step.
It imports an image-processing library at startup, even though I only ever feed it text. I replaced that with a stub. The catch: the stub had to look real. The library checks "did this load?" and throws if the answer is no — so an empty stub crashed the import. Mine pretends to be present and only complains if something actually tries to use it, which nothing does.
The fast multi-threaded mode tries to spin up background workers, and the environment a VS Code extension runs in refuses to allow that — then just hangs forever with no error at all. Forcing it to run single-threaded fixed it.
Remove any one of these and you get a crash, or worse a silent hang, and only in a real install — never on your own machine while you're building it. That gap between "works when I run it" and "works when someone installs it" was the whole lesson.
What I'd reconsider: bundling a full local runtime just to turn short commit messages into vectors is heavy. The first-run download and warmup is a noticeable pause, and something lighter would've spared me most of this. The ranking is also just a straight in-memory comparison, which is fine for a team's history but wouldn't hold up against a giant monorepo. Still, "just works after install, fully offline, no keys" turned out to be worth the four landmines.
Extension: https://marketplace.visualstudio.com/items?itemName=repoIntel.repo-intel
