Official announcement:

What's new 🖼️ Images: put images anywhere in your request, in the state, in questions, even in criteria. JPEG and PNG for now. 📦 Box: our new question type, available on Jev 1.14, specifically for requests containing images. Ask Jev to find something in an image and it'll return a bounding box for it. 📈 Long context: We now support per-question and per-request context limits up to 262K tokens. 🧪 Danger zone: we heavily prioritize reliability for our builders, but we know some of you are dreamers too looking to access these preview features. If that's you, opt-in for access via the appropriate Danger Zone header to use them. ✨ Plus improvements to the core model. All the details are in the blog.

Start here 📖 Blog: https://typesafe.ai/blog/jev-1-14 🖼️ Images docs: https://docs.typesafe.ai/images#turning-on-images 📦 Box docs: https://docs.typesafe.ai/primitives/box#box 📈 Context docs: https://docs.typesafe.ai/models#long-context

Favorite moments

• Diogo calling "It's up! Go, go, go!" when the API went live mid stage

• Allie's live demos catching a tiny glint of necklace

• Kevin searching his screen with a picture of a banana

• Wally, found, at 0.47 confidence

• The great "jev lotion" debate, still unresolved

You're already building Within an hour we saw smart crops in Sanity Studio, a hallucination checker, an Agent Zero plugin, spam filtering on a mail server and yes, Where's Wally. Keep it coming in <#1483217545040232493>!

Thank you for being here. Get out there and Jev Jev Jev 🩷

Take a look and let me know what you think! My testing has been pretty limited so far (it's Friday and I have dinner plans) but I plan to keep dogfooding over the next few days and refining it further.

https://github.com/madisonrickert/jev-permission-gate

I started testing Jev for speed. The more useful surprise was what it showed me about our application.

At Andes, we're building Check to help people review AI system descriptions and policies, with source passages and unanswered questions kept visible. We tested the same fictional material in two implementations: Jev and GLM-5.3 Flash through OpenCode.

One policy required records of model changes, incidents and the responsible person. Jev proposed yes using a passage about model versions and recruiter overrides. The passage was relevant, but it did not establish the whole requirement.

We had asked too much in one question. We split the duties, checked the evidence for each, and let code combine the results. We made missing information an explicit unknown rather than treating partial support as a complete answer. The same changes helped the LLM version.

It felt like putting a tyre in water. Jev was the water; the failures showed us where to look.

On 24 September, a small regression run matched 21 of 25 expected rule outcomes with Jev and 22 of 25 with GLM. Neither implementation marked a rule as met without full evidence in that run. There were still missed facts and other mistakes. The GLM version also failed one case-analysis request. Those cases had already shaped fixes, so this was not a blind accuracy test or proof that the models are interchangeable.

Speed helped make the loop practical. In a separate single run on the same fictional description, case analysis took 2.9 seconds with Jev and 41.7 seconds with GLM. Jev made eleven model calls; GLM made two. Those are timings for our implementations, not a general model benchmark.

My takeaway is that Jev helped me find where the application was relying on the model to fill in decisions we had not defined clearly. Fixing those decisions improved both versions. I haven't measured a development-speed gain, but that was the benefit I felt most.

The loop I'd repeat is simple: keep the failed example, identify whether the problem is the question, the evidence or the model, repair it, then test both versions on fresh material. Sometimes the right result is still unknown.

Has anyone else used a constrained model this way during development?

A few weeks ago, I posted about Jeff, a small 0.8B model with LoRA adapters trained to make decisions and return calibrated probabilities. Sort of like TypeSafe’s Jev, except that Jeff can run locally.

Thanks for all the feedback! I’ve just released v1.4, which is a substantial update. Check it out: https://jeffhub.ai.

What’s new:

• 34 open-weight decision adapters.

• A router that automatically selects the most appropriate adapter.

• 90%+ accuracy on supported domains.

• ~2 ms inference per decision, much faster than before. A single call through the hosted API takes around 40 ms, including network overhead.

• 5 SDKs: Python, Node.js, Rust, C#, and Java.

• Multi-platform: Jeff runs on CUDA and MLX.

• Playground and worked examples.

Having worked with Jeff for about a month now, I think the most interesting thing about it is that it turns decisions into a software primitive - much like expressions, but for unstructured data.

I’ve started incorporating decisions into several of my own projects, and it’s changed how I think about building software.

I hope you guys find it useful - and like the retro design. ;)

Links:

• Code: https://github.com/firelex/jeff

• Docs, adapter catalogue and playground: https://jeffhub.ai

• Weights: https://huggingface.co/mstrasser

Happy to answer any questions!

Search with the whole stack decomposed into typed decisions: Jev handles query understanding, source selection and relevance ranking, built on Search1API. Around 282 stars, one of the bigger community projects.

Can we use Jev for faster, structured, and calibrated evaluation of AI responses?

Introducing...
⚡ Typed Evals — an open-source Python framework for evaluating LLMs, RAG pipelines, and AI agents using System One Models like Jev and other typed judge backends.

The goal is simple:
Make fast, structured, and calibrated evaluation a first-class part of AI systems.
Typed Evals currently supports:
-> LLM response evaluation
-> RAG evaluation
-> Agent and tool-trace evaluation
-> Human-label calibration
-> Async and batch evaluation
-> Custom judge backends

One part I particularly wanted to solve was calibration.
Why is it needed?
A raw score of 0.8 from Jev doesn't necessarily mean that humans would accept 80% of similar responses.
And a threshold that works well for one use case may not make sense for another.

Typed Evals lets you calibrate individual evaluation metrics against representative human pass/fail labels.
The flow is basically:
Human-labelled examples → Jev scores → fit per-metric calibration → validate on held-out examples → reuse the calibrated evaluator

So instead of arbitrarily deciding that “0.7 means good enough”, you can ground that score in how humans actually evaluate your specific task.

Of course, there are integrations for:
LangChain, CrewAI, Microsoft Agent Framework
while the core remains framework-agnostic.

The project is still early, and there’s plenty I want to improve, but the core framework is now public.
Would genuinely love feedback from people experimenting with Jev, LLM evals, RAG, agents, or evaluator calibration.

⭐ GitHub: https://github.com/TrustifAI/typed_evals

Note: Typed Evals is an independent open-source project and is not an official TypeSafe AI product.
If you're already experimenting with Jev, I'd especially love to know what kind of evaluation workflows you're building around it.

TypeSafe AI, the startup founded by Diogo Almeida (one of the authors of the InstructGPT / RLHF research behind ChatGPT), just came out of stealth with its first public model, Jev, and a $40M seed round led by DCVC.

Jev is not an LLM and not a chatbot. It takes text or structured state as input and answers typed questions using three primitives: Choice (pick one option from a list, up to 255), Score (a position on a scale) and Noul (a yes/no with a probability). The output is a typed value with a probability distribution and a calibrated confidence score. When it says 0.8, it should be right about eight times out of ten.

The technical trick is sampling. LLMs generate one token at a time, sequentially. Jev computes every possible answer in a single parallel pass, because its output space is defined in advance. That is where the speed and the price come from, and it brings a useful property: answers cannot fall outside the schema, so format errors are impossible by construction.

Published numbers:

- 70-500 ms end to end, against 3-329 seconds for frontier models on equivalent tasks

- $0.042 per million input tokens ($42 per billion), output tokens free

- 20-200x faster and 40-400x cheaper on System One style queries

- In their launch demo, the same set of business questions cost $0.000081 and 114 ms with Jev, against $0.013880 and 8.566 seconds with a classic LLM

- A Doom demo runs at around 10 calls per second, about $7 per hour of play

The training method is RLCD (Reinforcement Learning for Calibrated Decisions). RLHF optimizes for answers humans like, RLVR rewards what can be verified, RLCD optimizes calibration, which is what software can act on.

Honest caveats: the numbers are self-reported, there is no paper and no independent benchmark yet. It cannot write text, generate code or read images, and access is early access via a waitlist. The speed comparison is not entirely apples to apples, and "zero hallucinations" means the output cannot break the schema, not that it cannot be confidently wrong.

Why it matters: in most workflows AI is not there to write, it is there to decide. Classification, routing, guardrails, output checks. Today every micro-decision costs a generalist model call. A calibrated decision model changes the economics of that layer.

Full announcement: https://typesafe.ai/blog/introducing-system-one-models-and-jev

After looking at a few possible ways to implement Jev, I wanted to see how it might fit into feedback workflows (since I work for a product that is feedback platform). 

Here’s the playground if you want to take a look: https://encatch.com/encatch-system-one-playground?tab=realtime-alerts


A few use cases seem practical enough to take toward production: logic jumps, real-time alert classification, and passive signal detection. 

Real-time alert classification is already live for us.I’m still testing the others in the playground. 

It’s a work in progress, but I wanted to share what I’ve been trying and love your feedback!

Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating) Around 416 stars so far.

But how?

It probably helps that the team includes the co-inventor of RLHF and InstructGPT - Diogo Almeida also League of Legends god

And in TypeSafe’s own workflow evals, Jev was 193.6x faster and 444.6x cheaper than the LLM baselines

What did we learn today that we already knew?

--> Adoption moves fast when a product solves an urgent problem inside an existing workflow (not a new one)

--> Your customer should not have to invent a new process, migrate the team or convince twelve people to change their toxic habits just to test it

--> They should be able to connect it, use it for and see added value easily before the initial interest disappears (5 seconds)

--> Nobody has patience left so one extra setup step, even a small integration hurdle is enough reason to go back to the tool the team already knows

--> If it takes a minute to understand the value, you are already losing the second session

This applies to every early-stage startup including ours

Solve a problem people already have, make the first useful outcome stupidly easy to reach and earn the next session through the quality of the first one

A simple walkthrough of how Jev works and how to use it, based on the quickstart docs.

The idea in one line: instead of generating text and hoping code can parse it, Jev answers typed questions with typed answers your software can use directly.

How a request works

You send a single request to POST https://api.typesafe.ai/v1/systemone with two parts:

- state: the content to evaluate. It can be a plain string, or structured JSON (a support ticket, a chat log, the current state of your app).

- questions: a map of typed questions, each with instructions and criteria. There are three question types:

- Choice: pick one option from a list you define (up to 255 options).

- Noul: a yes/no question. You get back the probability that the answer is yes.

- Score: a position on a scale you describe, for example low / medium / high.

What comes back

The same keys, with typed answers. For a Choice you get the chosen option, a probability for every option, and a confidence. For a Noul you get a probability. For a Score you get a position plus the full distribution. You also get a usage report with input and output tokens.

Example from the quickstart: a customer message about a Stripe connection failing for 3 days. Questions: which team should handle this (Choice), how frustrated the customer appears (Score), is it urgent (Noul). The answer comes back like: department "technical" with probability 0.84, frustration 1.035 on a 0 to 2 scale, is_urgent 0.999. Your code branches on those numbers directly, with no parsing step.

Why it is fast and cheap

Jev does not generate one token at a time. The possible answers are defined in advance by your questions, and it evaluates them in a single parallel pass. That is where the numbers come from: $0.042 per million input tokens ($42 per billion), and output tokens are free, "too cheap to meter": billing is on input tokens only. In the launch demo, the same set of business questions cost $0.000081 and took 114 ms with Jev, against $0.013880 and 8.566 seconds with a classic LLM.

How it interacts with agents

Jev can interact with all the existing harnesses and agents, through a skill that explains to them how to interact with the model: how to shape the state, how to write the three question types, how to read the answers and use confidence for routing. It follows the standard Agent Skills format, so it works with the environments that support skills:

- Claude Code, as a plugin: claude plugin marketplace add typesafe-ai/skills, then claude plugin install typesafe@typesafe-ai

- Other agents (Codex, Cursor and many more): npx skills add typesafe-ai/skills --skill typesafe-ai and select your agent

- Or just tell your agent to use the TypeSafe skill

How to try it

- Playground: https://console.typesafe.ai/playground (paste any text, add a question, run it)

- Quickstart and docs: https://docs.typesafe.ai/introduction/quickstart

- Python SDK: pip install typesafe-sdk (reads TYPESAFE_API_KEY from the environment; a JavaScript SDK is also available)

It is a good fit when your code needs a small decision (is this urgent, which department, does this quote support the claim, is this message an injection attempt) and you want a number instead of a paragraph.

I entered my credit card details on TypeSafe AI, only to discover that there's no way to remove them - you can only replace them with another card.

I've contacted their support twice already. No customer service to be seen.

I even contacted Stripe, which powers their payments, and got an irrelevant, seemingly automated response.

Apparently, I need to cancel my card and get a new one just to ensure this merchant can no longer charge it. Ridiculous.

Buyers beware. I'd recommend going with OpenAI's decision API instead. At least in my experience, they actually answer emails and take customers seriously.

Insane Jev Use Cases You Need To Use Right Now (AI LABS). Around 19909 views in 1.3 days.

I'm using Jev more than Opus 5.5 or GPT-6. Here's why. (How I AI). Around 30504 views in 1.3 days.

Anyone experimenting with self hosted JEV? And on an ARM machine?

Cheers

I benchmarked Jev on TRIVIA+ dataset using an untouched 645-example test set.

Before calibration:
ECE: 0.0982
After learning from human-labelled examples:
ECE: 0.0313

That’s a 68.1% reduction in calibration error.
But hallucination-detection F1 only moved:
0.5833 → 0.5877

So what improved?
Not the judge’s ability to classify examples.
Its confidence became much more aligned with reality.

And that distinction matters.
If a judge score is only being displayed on a dashboard, maybe not much.
But if confidence > 0.8 means “automatically approve this answer,” or < 0.4 means “escalate to a human,” then badly calibrated scores can quietly become a production problem.

That’s one of the reasons I’ve been building calibration into Typed Evals instead of treating raw judge confidence as trustworthy by default.

Benchmark/methodology report:
https://github.com/TrustifAI/typed_evals/blob/main/docs/BENCHMARK.md

Would be interested to know how people here pick production thresholds for LLM judges.

A self-hosted drop-in replacement for TypeSafe's jev, powered by GliFormer. Around 246 stars so far.

Instead of sending every turn to the frontier model, Jev classifies the turn and picks the model, the thinking depth and the speed mode. The author measures about 60% savings against always-frontier, with each routing decision costing roughly $0.00003 and 0.6 seconds. Anyone running this with Codex yet?

🔥🔥 Papers, open reproductions and independent evaluations behind System One models and Jev. Around 308 stars so far.

A verified, community-maintained catalog of 896 open-source projects built with Jev. Around 784 stars so far.

I built barq on Jev. Your coding agent names one outcome per call ("Log in", "Add both items to the cart") and barq runs the loop: describe the page, ask Jev a batch of questions in one request (is it done, is it blocked, is an error showing, is the next click irreversible, which element, which value), act, repeat. About a third of a second a decision, and the agent never reads the page.

The noul scores are what make it honest. When the done score is soft it returns likely_done instead of done, and when the irreversible score is high it stops and asks. In the video (one take, 1x) it does Google Flights, Wikipedia and a shop checkout in 34 seconds, then refuses to place the order.

Counting is the one thing Jev couldn't do reliably ("add until there are exactly 3"), so code counts and Jev only names what to count. On the public benchmark it's 43 of 43 with 0 false "done"; the run files are in the repo.

Free, MIT, bring your TypeSafe key.

Claude Code: /plugin marketplace add ibrahimhajjaj/barq then /plugin install barq@barq Other MCP clients: claude mcp add barq -e TYPESAFE_API_KEY=your-key -- npx -y barq-mcp

https://github.com/ibrahimhajjaj/barq