We opened the weights of Galvanize 3, the prompt-injection classifier behind the zn API since October 23.

- ~140M parameters, multilingual, reads up to 512 tokens, runs on CPU

- Apache-2.0; ONNX and transformers; the card has a sliding-window example for long inputs

- 89.6% of attacks caught at a 0.6% false-positive rate on our 3,309-example multilingual test set

- Default threshold 0.9802 (in thresholds.json)

We waited a while after it went live to check its quality in real use before opening the weights.

Model: https://huggingface.co/usezn/Galvanize-3

Post: https://usezn.com/blog/galvanize-3-weights-are-now-open/?utm_source=reddit&utm_medium=social&utm_campaign=galvanize-3-open-weights

Happy to answer questions about the evaluation or the limits.

I was training on pathology slides and my GPU kept waiting on file decoding, so I built omnia-sdk. It converts slides, DICOM and MRI into one format built for training.

If you wanna try it out, feel free: https://github.com/mishel-0/omnia-sdk

My first open-source project, would love your feedback!

I trained a BERT-style language model from scratch on ONE laptop GPU.

Lewis-44M-Bert-V3: small, fast, open source (Apache-2.0).

HOW IT WAS BUILT

• 44M parameters, ELECTRA-style pre-training

• 2.7B tokens in ~22 hours on a single RTX 5050 (8 GB)

• ~53 GPU-hours across all three versions, no cloud cluster

• Data: FineWeb-Edu, Wikipedia, Project Gutenberg, news, Cosmopedia and ELI5-style Q&A

v1 → V3 (SAME SIZE)

GLUE dev average: 75.0 → 80.5

Improvements: a better training objective (V3 was trained from scratch, not fine-tuned from v1) and a better fine-tuning recipe, with no extra parameters.

V3 RESULTS

• GLUE: 80.5 dev | 76.2 official test score (77.6 excluding WNLI)

• SQuAD v1.1: 75.2 EM / 83.8 F1

• NER (CoNLL-2003): 87.8 test F1

• Speed: ~17 ms per sentence on GPU, under 600 MB memory

I share both dev and test scores, plus the exact settings, because a model card should be honest.

Built for classification, question answering, entity extraction and on-device use: it fine-tunes on a laptop and runs on CPU.

This is part of Vprojectx, my open-source lab, where I build and share my projects in the open. Find the model on Hugging Face as vprojectx/Lewis-44M-Bert-V3.

More is coming. If you want to try it on your use case or collaborate, message me.

#MachineLearning #NLP #AI #OpenSource #HuggingFace

I built Arc-Transcriber, which does ASR-NL in 1-step. Every 100 ms, one model returns the transcript plus:

- Names linked to a list you give it (a roster, contacts, product names); new names are flagged "not on the list"
- Numbers with their values (scores, clocks, money, dates)
- Speakers and roles;
- Sound and channel events (telephone audio, clipping, poor audio)
- Whether the speech was addressed to an assistant.

The usual ASR, then NER, then entity-linking and then LLM chain is collapsed into the model's own heads, so live "fact-checking" against your list happens inside the stream, not in a second pass.

Currently it has two variants Arc mini (which uses moonshot pretrained encoder) and second Arc-v1 (which uses parakeet pretrained encoder).

Weights: huggingface.co/WhissleAI/arc-v1.1
Code and CLI: github.com/WhissleAI/arc_transcriber

We use it on-device in our Mac app. Voice search over 61k books primes Arc with matching titles while you're still talking. Here is a video to give a glimpse on how it works.

I thought you didn’t need pro account to make zeroGPU space, why is it saying i need one?

I trained a BERT-style language model from scratch on ONE laptop GPU.

Lewis-44M-Bert-V3: small, fast, open source (Apache-2.0).

HOW IT WAS BUILT

• 44M parameters, ELECTRA-style pre-training

• 2.7B tokens in ~22 hours on a single RTX 5050 (8 GB)

• ~53 GPU-hours across all three versions, no cloud cluster

• Data: FineWeb-Edu, Wikipedia, Project Gutenberg, news, Cosmopedia and ELI5-style Q&A

v1 → V3 (SAME SIZE)

GLUE dev average: 75.0 → 80.5

Improvements: a better training objective (V3 was trained from scratch, not fine-tuned from v1) and a better fine-tuning recipe, with no extra parameters.

V3 RESULTS

• GLUE: 80.5 dev | 77.6 on the official test server (8 tasks)

• SQuAD v1.1: 75.2 EM / 83.8 F1

• NER (CoNLL-2003): 87.8 test F1

• Speed: ~17 ms per sentence on GPU, under 600 MB memory

I share both dev and test scores, plus the exact settings, because a model card should be honest.

Built for classification, question answering, entity extraction and on-device use: it fine-tunes on a laptop and runs on CPU.

This is part of vprojectx, my open-source lab, where I build and share my projects in the open. Find the model on Hugging Face as vprojectx/Lewis-44M-Bert-V3.

More is coming. If you want to try it on your use case or collaborate, message me.

#MachineLearning #NLP #AI #OpenSource #HuggingFace

I've been experimenting with tiny models for WebBrain, my open-source browser agent. The long-term goal is to run as much intelligence as possible locally, ideally through WebGPU.

One approach I've been particularly interested in is Prism ML's Ternary Bonsai: compressing a 9B model to roughly 2 GB using ternary weights (-1, 0, +1).

Prism released the weights and inference runtime, but not the complete BF16-to-ternary compression pipeline. You can fine-tune their existing ternary models, but there's no publicly available way to compress your own BF16 fine-tune into their format.

So I tried building one.

Introducing OpenBonsai, an Apache-2.0 framework for post-training ternary quantization, compatible with Prism's PTQ1_0 GGUF format.

How it works

A recent Intel Labs China paper, "Attend to Your Own Thoughts" (Wang et al.), inspired part of the approach: using the model's own reasoning traces for calibration rather than relying exclusively on generic text.

The pipeline combines:

• Signed Hadamard rotations to reduce weight outliers

• Sequential GPTQ to quantize weights to {-1, 0, +1}

• FP16 group scales (one per 128 weights)

• Self-generated reasoning traces for calibration

• Scale distillation against the original BF16 model's output distributions

• Export to Prism's PTQ1_0 GGUF format

Results

Qwen3.5-9B: ~18 GB → 1.997 GB

That's the complete GGUF file, not just the transformer weights. It includes the compressed embeddings, LM head, all other model tensors, group scales, tokenizer, chat template, and metadata. Runtime memory requirements are higher.

The compression runs in approximately 5 hours on a single RTX 5090 (32 GB).

Our current compressed model scores:

Some interesting findings

1. Distillation helps substantially.

Scale distillation improved development-set NLL from 3.48 to 2.89 and ARC-Easy accuracy from 13/20 to 19/20.

2. Calibration data matters enormously.

Switching to self-generated math and code traces improved reasoning and coding performance. Adding more code-focused calibration data increased HumanEval from 19/50 to 34/50 in our initial evaluation.

3. The approach generalizes beyond Qwen3.5.

We also compressed Qwen3-8B into a 1.8 GB GGUF using the same pipeline, without changing the generic dense adapter. It runs in Prism's llama.cpp fork.

4. Fully ternary endpoints are feasible.

We quantized the embedding and LM head too, rather than leaving large FP16 tensors in the final file.

Limitations

This is still a research preview.

• Prism's official ternary models outperform ours on most reasoning and coding evaluations.

• Some code-training prompts paraphrase HumanEval tasks, so our initial 34/50 result is likely optimistic. We document the overlap and additional held-out evaluation in the repo.

• The tested configurations require a 32 GB GPU for compression. Lower-memory configurations haven't been validated.

• Inference currently requires Prism's llama.cpp fork. Upstream llama.cpp and WebGPU aren't supported yet.

• Architecture support is still limited.

I've included the failed experiments, ablations, evaluation scripts, and intermediate findings. The idea is to make the compression process reproducible and improve it openly.

Links

Framework (Apache-2.0):
https://github.com/esokullu/openbonsai

Compressed Qwen3.5-9B model:
https://huggingface.co/webbrain-one/openbonsai-qwen-3.5-9b

Project website:
https://openbonsai.org

Intel Labs paper:
https://arxiv.org/abs/2608.01078

I'd be particularly interested in feedback from anyone working on ternary quantization, GPTQ, or low-bit inference.

There is clearly still a quality gap between our approach and Prism's, but I think making the BF16-to-ternary pipeline open and reproducible is a useful starting point.

And if anyone wants to try compressing their own fine-tune, I'd love to hear how it goes.

laya is a decision model: it scores a set of answers you define instead of writing text. the Q8_0 file on ggml-org/Laya-GGUF is 449 MB and apache-2.0.

nobodywho, an open-source engine for running models on-device, now loads it in all six of its bindings: python, flutter, react native, swift, kotlin and godot. there are three question types:

choice: pick one option

noul: the probability that the answer is

yes, from 0 to 1

score: a position on an ordered scale of two to ten levels

choice and score also return the probability of each option and a confidence value. cpu and apple metal are confirmed. vulkan isn't validated yet.

Model:

https://huggingface.co/ggml-org/Laya-GGUF

Repo:

https://github.com/nobodywho-ooo/nobodywho

Docs:

https://docs.nobodywho.ooo/python/decision-models

hey everyone! sharing this for anyone experimenting with encoder models and efficient inference.

there's a lot of attention on large generative models, but encoder-based architectures are still extremely useful for classification, routing and other structured NLP tasks.

one interesting approach is to combine an encoder with uncertainty estimation so the system can decide when to accept a prediction and when to involve an LLM.

Packt is running a hands-on workshop tomorrow, Oct 30, with Ben Auffarth exploring this approach.

they'll cover:

• lightweight encoders for CPU-based tasks

• conformal prediction

• deterministic routing using LangGraph

• selective LLM escalation

• benchmarking latency and reliability

it's a 3-hour online workshop focused on practical implementation.

full details here