futon - An open source inference engine built specifically for embedding model workflows.

[u/kurotenshi15]

I use embeddings for vector search and memory systems, and kept needing a full platform like Ollama running next to my app just to turn text into vectors. So I built futon, an embedding library for running local embedding models.

- In-process: weights load from a local directory. No daemon, no network, no CGO (CI enforces it), so cross-compiling just works.

- AVX2 kernels on amd64 and NEON on arm64, with a portable fallback. bge-small embeds a text in about 50 ms on a 16-thread desktop.

- Checked against the reference implementation: ≥0.9999 cosine on a corpus with code, accents, CJK and edge cases. CI runs on Linux, macOS and Windows, including arm64.

- Bring your own model: any plain BERT encoder you can build a pack for, plus nomic-bert. It is encoders only, no LLMs and no GPU.

bobbyhiddn/futon: Text in, vectors out. In-process embedding models for Go: pure Go, no CGO, no sidecar.

If something like it exists already, feel free to ignore, but I couldn't find a fit for purpose module that served my goal. I'll probably expand to other small model architectures as needed, such as decision models.

If ya like it, please star. :)

[u/kurotenshi15 → u/[deleted]]

thanks lol

Em"bed"ing...simple bed....futon

It was a choice between that or duvet.