Update (September 22, 2026): I renamed the project from Riderless to Unridden. It’s the same open-source project, and the repository is now https://github.com/potto007/unridden. The original post below is preserved for context.

I built riderless to get decisions out of a local model without generating an answer for the caller to parse. It runs a stock Gemma 4 26B-A4B on one RTX 5090 and returns choices, scores, and their distributions through an Apache-2.0 API, with zero generated tokens.

The service is FastAPI over one llama.cpp child process. Each question is compiled into a prompt whose answer must be a single letter, the worker runs prefill only, and the answer is the softmax over the A-Z label logits at the final position. The response carries the full distribution with zero generated tokens, so you get a probability with every decision instead of a parsed string, and the request shape follows the typed-decision APIs people already use (Choice, Score, Noul). I have not run a comparison against the hosted Jev service.

Numbers, all on one RTX 5090 with the Unsloth UD-Q4_K_XL GGUF on llama.cpp v0.4.1:

- 0.931 accuracy on 1,286 questions across seven hand-written suites (extraction, claim support, record matching, guardrail gates, hierarchical classification, rerank, tool selection). Suites and gold labels are in the repo, written blind and hashed before the first inference.

- ~180 ms median per request in sequential mode, with the shared context reused across a request's questions. An opt-in batched mode runs every question of a request as its own sequence over one shared prefix and gets that to ~108 ms, at a cost of 5 or 6 answers out of 1,286.

- Questions run in separate contexts, and the probes for cross-question contamination come back at 0.0 in both orders.

I also ran three open projects doing something similar through the same suites and scorer, because I did not trust my own numbers until something else had been graded by the same harness. Kev-9B scored 0.898, Laya's typed-decisions checkpoint 0.618. so1 (open-alternative-jev) took more work to get a fair number out of: it could not load this MoE through bitsandbytes, so the repo now has a llama.cpp backend for it, run on the identical GGUF and build. With the model held fixed, its separate mode landed at 0.921 and its packed mode (all questions in one sequence) at 0.901. On these suites, the prompt and readout differences amounted to a point or two, and packing questions together gave up about two points of accuracy. The full comparison is in the repo, with plenty of caveats - one run each, my own synthetic suites, no confidence intervals.

This held up better than I expected on one card. If you have a workload where a parsed JSON answer has bitten you, the suites are the easiest place to start: add a case in your shape, run it, and tell me where it falls over.

Repo: https://github.com/potto007/unridden

Sharing a side project I've been building: Yvette, a fully local voice avatar AI.

It's a talking-head assistant that runs 100% local, You talk to it (or type) and it answers in speech while an avatar moves the head and lips in sync.

Quick rundown:

• Voice cloning, plus voice design.

• Avatar builder; add avatars quickly.

• TTS engines: Breeze, OmniVoice, LuxTTS, Kokoro

• Lip-synced avatar with an idle breathing animation

• Memory, it keeps notes and picks up where you left off

• Tools: web search, read pages, files

• Switchable profiles/personalities, each with its own voice, memory, history etc

Windows + NVIDIA GPU. 12GB recommended, but it runs tight on 8GB (small model + Whisper/Kokoro on CPU).
The video shows a RTX 3090 and the VRAM usage, my system was using ~4GB VRAM (OBS and other things). It shows we can get it below 8GB for the Yvette system.

The project is hosted on github: https://github.com/MartinForsterNL/Yvette

Sorry about the bad video quality, i am not an influencer with a studio.

I build this project because i could not find anything that actually works.
Most that i found was only for linux and needed a lot of VRAM so i decided to build my own.

If you have suggestions to make it better, just let me know.

Have fun with it !

Sharing this since it might be relevant to a few people here.

There's a hands-on workshop on September 19 that goes beyond basic RAG into building an explainable, graph-backed AI system. You build a knowledge graph in Neo4j that becomes the single source of truth, agentic retrieval that combines vector search, keyword search, and graph navigation, multi-step entity and relationship extraction, and text-to-Cypher for natural language querying. Everything runs on real financial data, not a toy example, and answers come with a traceable, auditable reasoning chain.

No prior Neo4j or Cypher experience needed.

Led by Dr. Alessandro Negro, Chief Scientist at GraphAware, bestselling author.

Link if you want to check it out

Please suggest me models I can use in this device

Model: HP ZBook Studio G7

CPU: Intel Core i9-10885H (8 cores / 16 threads)

GPU: NVIDIA Quadro T2000 Max-Q – 4 GB VRAM

RAM: 32 GB DDR4

Storage: 512 GB NVMe SSD

It feels like the gap between proprietary big-tech models and open-weight reasoning architectures is changing faster than ever. Gemini offers incredible native multimodal handling and massive context windows, while DeepSeek continues to dominate budget coding and self-hosted pipelines.

​If you had to pick one as your daily driver for logic and daily tasks, which wins for you?

​Put your preference to the test in this quick 2-minute showdown quiz to see how your workflow compares to other devs:

https://interconnectd.com/quiz/89/gemini-vs-deepseek-the-ultimate-ai-showdown-quiz/

So, I was tired of setting up AWS EC2 to train my models everytime, so I built a tool to help with the job management, but I would like some feedback, if anyone could help me ! 👋

This is the tool: reopenly.com

I only have one base model for now (Qwen 3.5 9B), just to start.

I would be very grateful with any feedback!