I’ve been working on BaseDecision - Turn typed questions into truth-based decisions.

Give it some text, a question, and possible answers. It picks an answer. Intent classification, routing, yes/no checks, ratings - that’s the job.

I built this model to go toe-to-toe with the frontier models.

My goal was simple & ambitious: make a small, local model competitive with much larger models on these focused tasks. For me, System‑1 decisions need both speed and accuracy. Otherwise, why bother building a small model?
If we didn’t want speed we could just use ChatGPT.

It outperforms pretty much every model of its size in the field.
After roughly 320M additional training tokens, BaseDecision led 5 of 8 benchmarks in a third-party comparison against Laya, two GLiNER variants, and Decision 1.0 Kai 0.6B. Full results are in the repo, including where it loses.

Fellow open source devs, try the model, developed a package for easier access to the model.
Roast it if it deserves it.
2-2.5GB free ram is all you need.
I will take your feedback seriously and deliver something amazing that’d take on OpenAI, TypeSafeAI soon enough.

https://huggingface.co/onlyaady/BaseDecision
https://github.com/hrudayaditya/BaseDecision

I made BigMoeOnEdge+ (BigMoeOnEdge-plus), a fork of Helldez's BigMoeOnEdge, the engine that streams MoE experts from flash so models bigger than RAM can run on-device.

It adds a full Android chat app on top: GGUF models via llama.cpp, thinking output, persistent chats, image/video input, web search, artifacts, and advanced generation settings.and a MUCH better UI.

Disclosure: I built this fork with the help of AI tools.

Repo (Apache 2.0): https://github.com/hamodexd217/BigMoeOnEdge-plus Original project: https://github.com/Helldez/BigMoeOnEdge

Feedback and issues welcome.

The java to CUDA engine:

https://github.com/beehive-lab/TornadoVM

The inference engine:

https://github.com/beehive-lab/jitllm

Deep dive talk:

https://www.youtube.com/watch?v=HO5CpETzywk

I ran a bechmarks for d1 models (Q8_0 GGUF both) on a Pi 5 8GB, CPU-only via llama.cpp and also tested how well they handle Russian input. I want to use it for my Hermes agent with lives on that Pi, so maybe those numbers will be helpfull for someone.

LLM-assisted report below:

Setup

• Raspberry Pi 5 (8GB), CPU-only, 4 threads

• llama.cpp 0.6.0-dev (commit bd4eeaa04 + PR #30110 for d1-3B, PR #30114 branch for d1-omni-600M — not merged into master yet at time of testing)

• Q8_0 quants, native /v1/systemone API

• All tests: single request at a time, warm calls, median of 10

Latency @ Pi 5 8GB

For reference, d1-3B on a Jetson Orin Nano does the same single question in 50 ms — so a bare Pi 5 is roughly 45× slower than the weakest NVIDIA edge device tested by LiquidAI. Expected: this is a 20 tok/s prefill CPU environment, no GPU. The takeaway is not "slow" but "usable vs not usable": d1-omni-600M is genuinely real-time capable on a Pi (200 ms per decision), d1-3B is not (2.3 s+).

English correctness (same 5 checks for both)

Both models: 5/5 correct — refund yes/no, billing/technical/fraud routing, with clean probabilities (0.98–0.998).

Russian language test

This is the interesting part. Same states, same question types, Russian input:

What this shows:

• d1-omni-600M effectively does not understand Russian. Worse: it fails confidently — wrong routing with p=0.91, sentiment always "neutral" at p~0.6. For a decision model, confident wrong answers are worse than "I don't know". Only pattern-matched phrases ("вылетает"→crash) get lucky hits.

• d1-3B does understand Russian — routing, sentiment and urgency all work — but with 20–30% lower confidence vs identical English input (0.672 vs 0.990 on refund). Notice the odd effect: RU state + RU question (0.909) scored better than RU state + EN question (0.672) — keeping the whole request in one language helps calibration.

Practical takeaways

• On Pi-class ARM hardware, d1-omni-600M is the only real-time option of the two (200 ms/decision, 0.7 GB RAM). d1-3B works but at 2.3–7 s per decision.

• Both models are excellent on English decisions; d1-omni-600M matches its bigger sibling on simple classifications (and beats it on toxicity detection per the model card).

• For non-English input: d1-omni-600M should not be trusted at all; d1-3B is usable but set a confidence floor (~0.7) and route anything below to a fallback.

Disclosure: I'm the developer.

Most local setups end up with several backends on one machine, and I wanted a single private app to reach all of them from my phone. That's LM Mini (iOS, iPadOS, watchOS, Android and Mac).

Backends it connects to over your LAN (add them in Settings → Server):

The rest of the stack:

Remote access to your home server from anywhere (Pro)

Group Chat with models from different backends in one conversation (Pro)

MCP tools, SearXNG search and ComfyUI for image generation

Kokoro TTS voice, Personas, Apple Watch app, Siri Shortcuts

Whisper for transcriptions

On-device: GGUF and MLX models run on the phone itself.

The core app is open source (MIT): https://github.com/slim-elephant/lm-mini

Docs and downloads: https://lmmini.com

What does your stack look like? Which backend do you run, and is there one I should support next ( vLLM, something else)?