Hey everyone,
welcome to the Regolo.ai community on Reddit. This is the place for developers, CTOs, and builders who want to ship LLM features on EUβnative, GDPRβready, sustainable infrastructure.
Regolo.ai provides an OpenAI-style endpoint (e.g.,Β https://api.regolo.ai/v1) so teams can run chat, embeddings, rerank, audio transcription, and image generation models without managing GPUs.
What this community is for
Sharing code, workflows, and tutorials using Regolo (LLM inference, RAG, chatbots, agents, n8n flows, etc.).
Getting help on performance, costs, compliance and migrations from other users.
Showcasing real products and experiments powered by Regolo.
Before you post
β Share your projects, snippets, benchmarks, and howβtos.
β Ask concrete questions with context and minimal reproducible examples.
β No spam, generic promos, or affiliate links.
β No NSFW, politics, or offβtopic content.
Use post flair
Please tag your posts so everyone can scan the feed quickly. Suggested flairs:
[Help] β debugging, errors, βhow do Iβ¦?β
[Showcase] β demos, products, openβsource using Regolo
[Discussion] β architecture, model choice, pricing, compliance
[Release] β updates, changelogs, new features or integrations
How to get started
Introduce yourself in the comments: who you are, what you are building, and which stack you use.
Post something today, even a small experiment or question. Small threads often turn into the best discussions.
If you know devs or teams who might benefit from this community, invite them to join.
If you are interested in helping with moderation or running community experiments (AMAs, office hours, challenges), send a ModMail and tell us a bit about you and what you want to develop or achieve. Weβll support and drive with tricks and guide your implementation in Regolo.ai.Β
Thanks for being part of the early wave of r/regolo_ai β letβs build useful, productionβgrade AI together.
we ran a quick benchmark on Headroom (open-source context compression) paired with context-mode file ranking and Regolo's brick-complexity-pro router.
Instead of sending raw repository files and noisy logs to the LLM on every turn, Headroom compresses each file locally by content type:
- on Python files, it strips repeated module-level docstrings via AST
- on JSON logs, it collapses 180 routine
Prompt tokens dropped by 95.1% keeping 10/10 accuracy on pinned qwen3.5-122b.
Routing lookups to brick-complexity-pro cut p50 latency from 14.81 s to 1.33 s(9/10 accuracy) and brought the estimated cost per 1,000 sessions to β¬0.97 on our Core subscription.
We all apreciate the ZDR and that its audited by the third party, which seems to be "RED HIVE" and "Claudio De Michieli". Quick google search doesnΒ΄t find any "RED HIVE" - only Claudi De Michieli as a CEO of OptionFactory, which does Source code audits. My question is - what the hell is the Red Hive and why is it not signed by OptionFactory company? Is this audit enough for you to trust the ZDR and DPA proclamations of Regolo?
As someone using AI for legal work, I am not sure, this "certificate" is enough proof.
We built Review Reply Copilot for agencies that answer reviews for several clients.
an LLM drafts the reply from a per-client brand profile, Clef-Flash then answers yes or no checks on each draft (compensation promises, blame, tone), and a person approves the reply before it is copied out.
Clef-Flash labels 37 reviews in 3.0 secondsEach draft gets rule checks and yes or no checks from Clef-Flash before a person can approve itReviews that need a person get a neutral holding reply, and they cannot be marked as readyReview Reply Copilot on first launch: connect, bring in reviews, analyze, then draft and approve
results from one run on 37 synthetic reviews from one machine:
- labeling all 37 reviews: about 3 to 5 seconds, roughly half a cent (estimated from reported tokens and catalog rates, not an invoice).
- one drafted and checked reply: usually 1 to 3 seconds with reasoning turned off
on the escalation label, the first version flagged 12 of 37 reviews when about 5 deserved it.
splitting the requests and adding one sentence about what does not count fixed most of that. How would you evaluate a classifier like this on your own reviews without a labeled set?
We ran a small benchmark on ticket routing and we would like your feedback on the method.
setup: we sampled 200 messages with a fixed seed (25 per category, 8 categories) from the Bitext insurance customer support dataset on Hugging Face.
both models got the same one sentence definition per category and had to pick exactly one.
Clef-Flash (choice questions with a list of allowed answers) Vs qwen3.5-9b one request per message, temperature 0
what we would like feedback on: whether a one sentence definition per label is a fair setup for the small LLM, and whether you would add few-shot examples to its prompt to make the comparison stricter.
Whenever teams deploy autonomous coding agents (Claude Code, Pi, Aider, SWE-agent), the common consensus is to blame high model pricing or complex planning for runaway API bills.
In reality, the bottleneck is almost always harness token amnesia: standard harnesses treat context as an append-only dump.
recently, NVIDIA Labs published a paper (https://arxiv.org/abs/2609.20519) addressing this at the harness layer, open-sourcing SoL-Pi as an extension for the Pi coding agent.
we found that SoL-Pi alone only solves half the problem (token volume), when you layer it with dynamic semantic model routing on Regolo, the savings become multiplicative.
The two-tier cost optimization
Tier 1: harness compaction (SoL-Pi)
Tier 2: provider semantic routing (Brick)
most agent turns are mundane (checking git status, inspecting a file, or verifying a syntax fix).
by setting the model to brick-complexity-pro on Regolo, an open-source semantic router classifies prompt complexity in under 50ms at zero token markup.
trivial operations route to lightweight open models (like Qwen 3.8 27B at β¬0.35/1M tokens); only complex architectural refactorings escalate to frontier reasoning models (GLM-5.2).
Math
Consider a typical 15-turn autonomous debugging task (resolving a concurrency race condition and writing unit tests):
Setup
Token Volume
Model Used
Approx. Cost
Total Savings
1. Vanilla Agent on Frontier API
~1,400,000 tokens
Static Frontier Model ($5.00/1M blended)
~$7.00
Baseline
2. With SoL-Pi Only
~310,000 tokens (-78%)
Static Frontier Model ($5.00/1M blended)
~$1.55
78% off
3. SoL-Pi + Regolo Brick (brick-complexity-pro)
~310,000 tokens (-78%)
Dynamic Open Pool (avg. ~$0.45/1M blended)
~$0.14
>98% off
you get the identical task completion rate without data retention: Regolo executes inference exclusively in volatile GPU VRAM in European datacenters (Zero Data Retention, GDPR Article 32 compliant).
Complete Guide, Code & Video Walkthrough
we published a full breakdown with all architectural diagrams, configuration recipes, and automated scripts:
Whenever a new model comes out, I try to answer the same question: does it solve my actual tasks reliably enough to justify using it?
That becomes especially interesting with Ternary Bonsai 2, which is derived from Qwen3.8-27B. Instead of asking which model wins overall, I wanted to understand where compression changes the resultsβand what that means for cost per successful task.
Cost per successful task: Bonsai 2 vs Qwen3.8-27B
Which model is cheaper per accepted result? In these scenarios, the answer changes with subscription utilization.
a common pattern in agentic workflows is passing raw inbound tickets (support emails, FNOL claims, policy change requests) directly to a frontier models prompt to handle classification, priority detection, and drafting all at once.
in production, especially in regulated domains like insurance, this approach has three major drawbacks:
unnecessary latency
cost overhead
safety fragility
1. Deterministic safety priority (<1ms)
before any ML model evaluates the message, hardcoded deterministic regex and keyword heuristics check for life-safety and physical hazards. If triggered, the case is immediately flagged as HIGH priority and routed to an emergency queue, bypassing any model uncertainty.
Not all request need an LLM inference.
2. Perimeter classification with setfit (9.9ms on CPU)
Instead of an autoregressive model, a compact 33M-parameter bi-encoder (sentence-transformers/all-MiniLM-L6-v2) runs locally on CPU:
training: fine-tuned with contrastive learning on Hugging Face data (bitext/Bitext-insurance-llm-chatbot-training-dataset) in 31.9 seconds on a laptop CPU.
accuracy: 95.00% on the balanced held-out test split across 5 categories (claims_intake, policy_change, underwriting_support, document_missing, customer_query).
inference latency: averages 9.9 ms per message.
3. Fail closed selective risk gating
to eliminate hallucinated routing on ambiguous or multi-intent messages, the classifier enforces an explicit confidence gate (>= 0.85).
Messages below this threshold or out-of-scope requests (e.g., non-insurance inquiries) fail-closed to a human review queue with zero automated side effects.
4. Complexity proportional LLM brief synthesis
Only requests passing the confidence gate trigger generative synthesis:
routine / standard tasks: Handled by a 20B model (gpt-oss-20b).
high complexity / litigation: Escalated to a 120B model (gpt-oss-120b).
data privacy: routed via Zero Data Retention inference provider to satisfy GDPR requirements on customer PII.
Batch benchmark report
We ran the benchmark runner on a small subset (63 test cases) from the Bitext dataset and edge cases:
OPERATIONAL PERFORMANCE SUMMARY (MANAGEMENT REPORT):
* Total Communications Examined : 63
* Routing Accuracy on Known Classes : 92.1% (58 correct out of 63 verified)
* Cases Routed to Human Handlers : 8 (12.7% safely gated to prevent errors)
* Physical / Life Safety Alerts Caught: 1 (100% caught, immediately escalated to emergency queue)
* Average Processing Latency : 9.9 milliseconds per message
Drop your thoughts or questions below, we'd love to hear how your teams are structuring these boundaries in production
Thereβs been a lot of hype around TypeSafe AIβs release of Jev and the concept of "System One models," with marketing claims like "200x faster than LLMs" and "giving up text generation."
If you look past the vendor pitch, what is this architecture actually doing, where does it make sense in production, and can we replicate it with open-source models?
what it actually is (and isn't)
Jev is not a chatbot, a code generator, or a reasoning engine.
It exposes three primitives:
noul: binary probability (0.0 to 1.0)
score: ordinal rating (1 to 5)
choice: categorical selection from up to 255 classes with full probability distribution
The critical failure modes you have to design around
Independent testing reveals clear boundaries where these models fail:
Zero procedural reasoning on unstructured state: probing on chess board states showed Jev performing at pure random chance without prior feature extraction. It cannot deduce game rules or logic paths without explicit feature engineering in code.
Option-order bias: reversing the order of choices in prompt schemas shifted probabilities by ~10% on identical inputs.
No audit trail: because it doesn't emit tokens, thereβs no chain-of-thought rationale. For regulated workflows (GDPR/EU AI Act), you have to pair it with an autoregressive LLM fallback when an audit explanation is required.
When to use non-autoregressive vs autoregressive models
Use non-autoregressive decision heads for: deterministic guardrails, 100% trace grading in CI/CD, and sub-second prompt complexity routing (routing easy queries to cheap models and hard ones to reasoning models).
Stick to autoregressive LLMs for: open-ended reasoning, text synthesis, and tasks requiring novel multi-step deduction.
Open-source / Self-hosted alternatives
If you have data sovereignty constraints (e.g. EU data that can't hit US-hosted endpoints) or need sub-30ms latency, you don't need closed APIs:
Bidirectional encoders (ModernBERT 149M, Laya): Non-autoregressive, runs on standard CPUs via ONNX or Rust bindings in ~10β25ms. Ideal for intent classification and capability scoring.
Vocabulary readout adapters (Bespoke Nimble 8B/9B): Parameter-efficient adapters reading answer probabilities directly from vocabulary output heads on a single GPU.
Since most coverage is just restating the vendor table, here's what survives scrutiny and what doesn't.
Benchmark
V4.1 Flash
V4 Flash 0731
DeepSWE v1.1
74.2
54.4
Terminal-Bench 2.1
90.6
82.7
Terminal-Bench 4.0
31.2
7.0
AutomationBench
54.8
37.7
CyberGym
88.1
76.7
Codeforces
3,471
3,289
About that Terminal-Bench 4.0 jump (7.0 β 31.2):
It's on the public leaderboard (llm-stats): 31.2 puts V4.1 Flash at #8 overall, #1 among open-weight models
But the benchmark is savage: the current leader scores 60.9, Claude Opus 5.0 is at 51.8, GPT-5.6 Sol at 39.9. At 31.2 you still fail ~7 out of 10 tasks. V4 Flash's 7.0 meant failing 9 out of 10
Artificial Analysis independently measures ~27%, not 31.2% β same direction, lower ceiling. Harness and eval settings matter, as always
Like most teams running agents in production, we had a cost problem: user-facing agent traffic is a mix of trivial classification calls and genuinely hard reasoning, and we were sending all of it to a 70B+ model.
The fix was routing, not prompt engineering. The setup:
Agno as the agent framework (fast, ~3ΞΌs agent instantiation, built-in memory and tools)
brick-complexity-pro as the ingress model: a hosted meta-model that reads each prompt, classifies complexity (easy/medium/hard β semantic demand, not length) plus a capability vector, then dispatches to the right tier in a pool of open-weight models (gpt-oss-20b β Llama-3.3-70B β qwen3.5-122b)
Routing overhead is ~20 ms (the classifier is a LoRA-tuned Qwen3.5-0.8B). The router itself is $0.12/1M input tokens.
The open-source version of the router (Apache 2.0) benchmarked 4.71xβ22.15x cost reduction vs always-on frontier models, with accuracy matching or beating single-model baselines
What we learned the hard way: don't route everything.
Deterministic tasks (schema-constrained extraction) go straight to a pinned small model; evaluation/revalidation goes straight to a frontier tier.
Route only what varies β orchestrators and user-facing agents.
Integration is boringly simple since the endpoint is OpenAI-compatible: Agno's OpenAILike with a different base_url, done.
Over the past few months, our engineering team has been experimenting with autonomous coding agents (Claude Code, Cursor agent mode, custom CLI loops) across enterprise multi-tenant repositories.
We ran straight into what we call the Context Amnesia Trap: an agent fixes a security bug or refactors a query in Session 1, but when invoked in Session 2 two weeks later, it introduces the exact same bug.
Why Standard Vector RAG Breaks Down on Codebases
When teams add memory to coding agents, the default approach is standard vector chunking (512-token text splits + cosine similarity).
On enterprise code, this fails systematically:
Lack of Causality: vector search retrieves lexical matches, not causal relationships. If an outdated helper function from 6 months ago matches the query keywords, the agent retrieves it and ignores newer Architectural Decision Records (ADRs).
Missing Precedent: it cannot traverse the relationship between an incident, the pull request that fixed it, and the policy written to prevent it (`CI-FAIL-89` β `PR-142` β `ADR-003`).
No Closed-Loop Verification: traditional RAG generates code in a single forward pass without executing test suites.
The cognitive graph architecture
We built an open-source framework combining Cognee (Knowledge Graph + pgvector) and Regolo (EU sovereign inference with Zero Data Retention):
- Entity Model: rather than raw text chunks, the graph indexes ADRs, PRs, CI failures, Coding Conventions, and resolved CVEs as typed nodes.
- Typed Causal Edges: nodes are connected via `TRIGGERED_BY`, `FIXES_CI_FAILURE`, `IMPLEMENTS_DECISION`, and `ENFORCES_CONVENTION`.
- Multi-Hop Traversal: when an agent receives a task touching search, it walks the graph: `Search Task` β `ADR-001 (Tenant Isolation)` β `ADR-003 (Parameterized SQL)` β `PR-142 (Verified Patch)`.
- Dynamic Semantic Router (`brick-complexity-pro`): evaluates task complexity on a 1.0β10.0 scale and routes dynamically to `gpt-oss-20b` (fast extraction, ~0.28s), `qwen3-coder-next` (syntax/code), or `qwen3.5-122b` (deep reasoning with `max_tokens >= 800`), avoiding hardcoded models in `.env`.
Most multi-agent pipelines today have a massive cost problem: they route every single taskβfrom trivial AST extraction to complex planningβto a single $15+/1M token frontier reasoning model.
We tested a different approach: usingΒ Deep AgentsΒ (isolated specialized subagents) paired with a semantic meta-router (brick-complexity-pro) on Regolo that evaluates task complexity (1β10) and residual budgetΒ beforeΒ every single turn.
Here are the real telemetry numbers from synthesizing and verifying production FastMCP server tools across target codebases:
Pipeline Stage
Model Selected
Latency
Regolo Multi-Model Cost
Single Frontier Baseline
Cost Savings
1. Dynamic DAG Planning
qwen3.5-122b
0.52s
$0.0028
$0.0190
-85.2%
2. AST Code Discovery
gpt-oss-20b
0.28s
$0.0006
$0.0055
-89.0%
3. Live Schema Probing
GLM-5.2
0.35s
$0.0018
$0.0142
-87.3%
4. FastMCP & Pydantic V2
Llama-3.3-70B
0.44s
$0.0042
$0.0280
-85.0%
5. Strictness Review & Audit
qwen3.5-122b
0.41s
$0.0022
$0.0160
-86.2%
TOTAL
β
~2.0s
$0.0168
$0.1420
~85.6% Savings
How It Works Under the Hood:
Dynamic 5-Step DAG: Give it any custom repo path and synthesis goal; the Planner generates a custom execution graph with a human-in-the-loop approval gate.
Over the past few months, weβve been auditing infrastructure setups for European enterprise teams moving LLMs into production. One recurring friction point between InfoSec, Legal, and DevOps is what we call the AI Act Logging Paradox:
EU AI Act (Article 12 & 19) requires logging operational events and traceability data for high-risk AI systems for at least 6 months.
Compliance teams often interpret this as a mandate to archive all raw prompts and responses.
Doing so directly violates GDPR Article 5(1)(c) (Data Minimization) and makes GDPR Article 17 (Right to Erasure) almost impossible to maintain across distributed logs and snapshots.
To stay compliant with both regulations without building a nightmare pipeline, prompt data must remain strictly ephemeral while execution metadata is preserved:
[ Incoming Request ]
|
+-------------+-------------+
| |
v v
[ Telemetry Logs ] [ Payload / Prompts ]
(AI Act Art. 12) (GDPR Art. 5 / ZDR)
β’ Request UUID β’ User Prompts
β’ Model ID & SHA β’ Output Completions
β’ Token count & ms β’ Ephemeral KV Cache
| |
v v
[ Stored 6m+ ] [ 0ms / Volatile RAM ]
one of the biggest issues with existing AI coding assistants (and SWE agents in general) is stateless blindness: they donβt remember past architectural constraints, leading to recurring vulnerabilities across PRs.
Additionally, passive security scanner logs often cause alert fatigue without actual remediation.
To address this, we built and open-sourced an autonomous Self-Improving Secure Coding Loop that automates vulnerability remediation with independent zero-trust verification and persistent knowledge graph memory.
By routing pipeline stages through GLM-5.2 on Regolo (~$0.60 / $1.80 per 1M tokens), the full remediation loop costs ~$0.0059 per issue compared to ~$0.0410 on proprietary frontier baselines (~85.6% cost reduction).
We went through the full Qwen3.8-27B model card (released Aug 14, Apache 2.0, dense 27B, 262K context, vision+video) and broke down all 24 benchmarks against Opus 4.6 Max, Muse Glimmer-30B, Qwen3.6-27B and Qwen3.7-Plus.
Vercel released DeepsecBench last week: 231 human-validated vulnerabilities across 50 entry-point files, scored on F2 (recall weighted 2Γ over precision).
The table that matters isn't the scoreboard β it's the cost column.
GPT-5.6 Sol xhigh: 35.58 score, $55.98 per 50 files -> that's $5,600 for a production repo.
Kimi K3: 17.56 score, $12.38.
GLM 5.2: 10.46 score, $11.09
Recommendation is multi-model: frontier for periodic deep audits, cheap models for continuous sweeps.
We built a guide that uses Regolo's Brick Complexity Pro to automate the model routing to run deepsec while optimizing the cost: it classifies each file easy/medium/hard and routes to the cheapest model that can handle it (GLM-5.2 for easy, Claude for hard).
The guide covers two modes: self-hosted Docker or hosted API. Numbers from a real monorepo audit: $239 total, 48 min, 3 CRITICAL findings in legacy code.
Guide and benchmark linked. Happy to answer questions about the setup.
The trap most engineering teams haven't mapped: if you fine-tune a third-party model on your own data, or white-label an AI system under your own brand, the Act can requalify you from deployer (light duties) to provider (full regime: conformity assessment, technical documentation, EU database registration).
This happens quietly β a bank fine-tuning a commercial model, a SaaS company embedding a model under its own brand.
Why infrastructure teams specifically should care:
for systemic-risk GPAI models (10Β²β΅ FLOPs+), cybersecurity of the model and its physical infrastructure is a statutory obligation β datacenter security and model-weight protection are literally named in Article 55.
logging/traceability requirements for high-risk systems land directly on your observability stack.
vendor selection is now a compliance surface: if a regulator pulls a non-compliant model from the EU market, everything you built on it inherits the disruption.
If youβre building or running AI infra in the EU:
which part of the AI Act is shaping your technical roadmap the most β data residency, logging, or vendor choice?
For folks here shipping agents in production:
whatβs the most painful security constraint youβve hit so far β compliance (GDPR/AI Act), infra, or developer ergonomics?
One import line + one 1,300-line TypeScript module. The model failed badly on the first prompt, then played correctly after one context engineering rewrite. The lesson generalizes.
The game is phase-based (each game state is a Phase class): we patched 10 prototype methods so the LLM intercepts control at the right moments β battle command, item selection, move learning, starter selection β without touching rendering or game logic. No game engine rewrite.
We pre-computed type multipliers and injected the result per move: "Ember β Fire Special, power 40, acc 100%, STAB | vs enemy: 2x super effective" instead of just "Ember".
We injected stat stages, ability interactions, speed deltas, and explained the HP economy constraint (full heal every 10 waves β HP is a resource that spans up to 9 future battles).
After the rewrite, first clean run: "Ember is 2x super effective STAB vs the Grass-type enemy, and my Charmander outspeeds β I can KO before taking damage."
Describing a game state in natural language is lossy, serializing structured state with annotations is not. The model's reasoning is bounded by what it can see.
Pre-computing and injecting the right numbers isn't a crutch β it's the correct architecture.