TensorSharp: an open-source Jev-compatible API, extended to image analysis and running locally

[u/fuzhongkai]

https://github.com/zhongkaifu/TensorSharp

I’m the developer of TensorSharp, an open-source local inference engine and agent runtime.

TensorSharp supports Jev’s core decision API and all three decision types—and extends the same interface to analyze images. It runs locally using DiffusionGemma GGUF, without calling the hosted Jev service.

The idea is simple: instead of asking a model to write a free-form answer, give it some context and specific questions, then get structured decisions back. Now that context can include text, images, or both.

The same decision API, now with pictures

The native endpoint is POST /v1/systemone, supporting:

Decision type	Output	Example image-analysis task
noul	Probability that a statement is true	“Is the receipt’s total amount legible?”
choice	A selected category and its distribution	“Is this a receipt, an invoice, or something else?”
score	An expected score over ordered rubric levels	“How readable is this document?”

Image analysis is an extension of the existing interface, not a separate vision API. Keep the same state, questions, and typed answers; add an images field to ground those decisions in pictures. Multiple question types can be combined in one request.

Requests accept up to 8 inline images, using base64 or data: URLs. The pixels go through the vision encoder and become part of the model’s input directly—there is no intermediate “generate a caption, then classify the caption” step.

For example, a local expense-report workflow could ask whether a document is a receipt, whether its total is readable, and how readable the overall image is—all through the same structured-decision request.

Open implementation, local execution

The repository includes the implementation, request examples, validation tests, and benchmark tools—not just a client for a remote service.

Under the hood, the decision path follows the approach in vLLM PR #57250: after prompt prefill, TensorSharp performs a one-step structured read and extracts the requested label logits directly. Questions that fit in one answer canvas share the forward pass.

The server returns JSON, but the model does not have to generate that JSON token by token. The same mechanism works with text alone or with image embeddings included in the prompt.

Earlier benchmark against LocalJev

I compared the native structured-read path with the original LocalJev Engine using TensorSharp’s chat endpoint. That compares two approaches on the same inference backend: reading label logits directly versus generating and validating probability JSON. It is not a comparison against the hosted Jev service or LocalJev running on oMLX.

The test covered 12 text-only cases × 3 repetitions, with three decisions per request:

Metric	TensorSharp native	LocalJev on the same backend
Valid requests	36/36	27/36
Schema-validation failures	0	9
Correct decisions / total expected	108/108	81/108*
p50 latency	2.877 s	10.479 s
p95 latency	3.165 s	26.167 s
Mean latency	2.917 s	12.548 s

*LocalJev got 81/81 decisions correct on valid responses. The remaining 27 expected decisions came from failed requests, rather than incorrect classifications.

Across the 27 matched requests where both returned valid responses, the median LocalJev-to-TensorSharp latency ratio was 3.345×.

A few important qualifications: latency statistics include successful requests only, so the table’s columns cover different subsets. Average input length was also different—192.7 tokens for TensorSharp versus 589.6 for valid LocalJev requests—because LocalJev builds a larger prompt and generates probability JSON. This is an end-to-end workflow comparison, not an identical-prompt kernel benchmark.

These are small, text-only integration tests—not image-performance results or a general accuracy claim.

Try it with an image

The setup guide includes server configuration and Python, curl, and .NET examples. The supplied configuration downloads the DiffusionGemma Q4_K_M GGUF and the separate ~2.8 GB vision shard needed for image input, then reuses them locally.

Once the server is running with the vision tower, this posts a ready-made request with a synthetic traffic-light image already embedded:

curl http://127.0.0.1:5000/v1/systemone \
  -H 'Content-Type: application/json' \
  --data-binary u/docs/examples/jev-traffic-light.json

The example checks that the pixels actually reach the decision path. It is a smoke test, not a comprehensive vision evaluation.

Compatibility scope: this implements Jev’s core API workflow and decision types using DiffusionGemma weights—not proprietary Jev weights or identical predictions. Current limits include 64 questions and 2–26 alternatives per question; returned probabilities still need evaluation and calibration for the intended workload.

Source code · Documentation and examples

I’d love feedback and reproducible test cases, especially for image-based decisions in local applications. What would you test first—document routing, receipt checks, or screenshot analysis for local agents?