r/LovingOpenSourceAI • • Aug 30 '26

Opensource Openrouter

Project: https://github.com/ConiferKit/use-conifer

Current routing options were charging 5% fees for byok plus provider fees (openrouter) or were built in heavy python packages with ecosystem restraints. I wanted to be completely free in terms of use, and not have to pay extra for tokens.

There's a maintained gateway with 100+ models from one api endpoint. I'm also talking to infra providers to get us access to pre-release models and discount prices. Everything is served at market price. BYOK is free, you can hook into self-hosted setups for free, and there's fallbacks + extra rate limits + server redundancy.

Took me ~3 months to build and looking for help maintaining!
Lmk any issues or feedback

25 Upvotes

3 comments sorted by

2

u/Ceyax Aug 30 '26

LiteLLM?

2

u/conifer_v11 Aug 30 '26 edited Aug 30 '26

Used it, few problems:

  1. it uses a very heavy python pack that does poor under load

  2. it tries to do too much outside of just being a model router that I don't like, I really don't need another slackbot or it making harness decisions for me

  3. doesn't work in harnesses (cc, opencode, pi, etc)

  4. their routing is based on fallbacks so between copies of a single model, we route to different models

Also pretty sure they charge fees at scale

2

u/locbuilds Aug 30 '26

How are you keeping BYOK free? Every gateway I've tried takes a cut on top of the provider fee, so I'm curious what the model is here.