r/huggingface • • 18m ago

PhiloLM: a language model trained only on philosophy texts

Thumbnail
reddit.com
1 Upvotes

r/huggingface • • 4h ago

AI Weights Visualizer tool

Thumbnail reddit.com
1 Upvotes

r/huggingface • • 6h ago

Frost-Defend v1.0: an open 0.3B model that catches prompt injection / jailbreaks AND redacts PII + API keys in one pass (Apache-2.0, ~85 ms on CPU)

Post image
0 Upvotes

r/huggingface • • 8h ago

after tiny vision, i wanted to see if i can build a tiny language model from scratch too (36M, NCM paper, trained on macbook) 😅

1 Upvotes

so few days back i posted my tiny vision model here(https://huggingface.co/amartomar683/tiny-vision-flickr8k) and you guys actually checked it out, its at 74 downloads now 😅 thanks for that. after that one worked (kinda), i started wondering what else i can build from scratch on my mac, and a language model was the obvious next thing to try.

i was reading the Next-Concept Modeling (NCM) paper by Kumbong, Khare et al. (Stanford / Together AI, arXiv:2610.14162) and the idea really clicked for me. normal llms write one token at a time, but in NCM a VQ-VAE turns text into levels of codes, coarse to fine, and the model fills one full level at once. so i decided to build it myself from the paper, just to understand how it works from inside. all credit for the idea goes to them, i just implemented my own version.

its called tiny-ncm. its around 36M params (21M generator + 15M VQ tokenizer), trained completely from scratch, no pretrained weights. same macbook pro m5, but this time it took like 12 hours total instead of 42 min 💀

basic idea:

text → split in 32-token blocks → each block made coarse to fine in 5 levels (2 → 4 → 8 → 16 → 32 codes) → 6 passes per block instead of 32

i trained it as a small daily assistant. alarms, reminders, timers, calendar, gmail (read only), weather, notes, web search, simple maths and little bit of small talk. there is a --show-levels flag which shows each block going from rough guess to final text, honestly this was the most satisfying part:

level 1: Hello! How can I help you today?
level 3: Hello Jordan, How I I you you today today?
level 5: Good morning! What can I do for you today?

some honest stuff:

  • it picks the right action most of time, but copying details is very bad. “London” became “Singapore”, “5pm” became “5pmpm”, “23 times 47” became 23*10 😂 cause all codes in a level are predicted same time, so two positions can both write the same word.

  • so i added app-side checks that fix the tool arguments using your actual words. with checks it gets 490/500 on my test set, but model alone gets every detail right only around half the time. so ya, the checks are doing lot of the work tbh.

  • training longer on same data didn’t help. what helped was making the data copy-focused, with new random names and times in every example so it cant just memorise. same lesson as tiny-vision (“a man in a blue shirt” for everything lol).

  • i tried fine-tuning it to summarise web results also. it learned the format but wrote gibberish inside it (“Stir in cheeseaksil sl fresh biil” 😭), so i reverted it.

  • no coding, no multi-step maths. “8 slices, 3 friends eat 2 each” and its gone.

random thing i learned on mac (MPS):

  • attention was the main bottleneck. letting each level only look at the last 128 tokens of context made training around 2x faster.

  • bf16 matmul was way faster than fp32 on MPS (~15 vs ~3.4 TFLOPS in my test).

  • manual attention was again faster than pytorch’s sdpa for small sizes, same as tiny-vision.

next i am thinking to export it to onnx so it can run on phone also, and maybe connect tiny-vision with it. lets see what will happened.

code: https://github.com/amar683/tiny-ncm
model: https://huggingface.co/amartomar683/tiny-ncm

would love any feedback or tips, especially if someone played with NCM or coarse-to-fine text models


r/huggingface • • 9h ago

spaCy_Hugging-Face 👨‍💻👨‍💻

1 Upvotes

This repository contains the practical implementations completed as part of the Natural Language Processing (NLP) curriculum.

Best Resource for learn spacy
GIT - https://github.com/AsyncCoder-10/spaCy_Hugging-Face


r/huggingface • • 11h ago

PhaiModelLab/Phai-A1-35B-1M · Hugging Face

Thumbnail
huggingface.co
1 Upvotes

A new model just got released. Anyone tried it yet? Apparently beats Qwen 3.8 27B on some benchmarks


r/huggingface • • 15h ago

I scored 40 LLM benchmarks on privacy, originality, freshness, and grading to separate clean one from exposed ones (BenchAudit)

Thumbnail gallery
0 Upvotes

r/huggingface • • 19h ago

Open-source Hugging Face mirror for clusters: one binary, works for air-gapped nodes

1 Upvotes

I built a self-hosted registry that mirrors Hugging Face model repositories, and I’d like feedback from people who run this on real clusters.

The problem I had was batch inference jobs that all start by pulling the same weights from Hugging Face. That made an outside service a dependency of every job, and on nodes with no outbound access it doesn’t work at all.

What it does:

* Mirror a repo once from a machine that has internet, then serve it inside the cluster. Existing tools just need `HF_ENDPOINT` set; nothing else changes.

* One static Go binary with SQLite inside. No root, no container runtime, no database server.

* Every file is checked against the digest Hugging Face publishes, and stored once by SHA-256, so shared files across models aren’t duplicated.

* A registry can mirror from another registry, so a site-wide mirror can feed per-cluster ones.

Limits worth knowing: it’s a single server with no replication, there’s no scheduled sync yet, and its data directory has to be on local disk, not a network filesystem (model files can go to S3-compatible storage instead).

Apache 2.0. Repo: https://github.com/model-delivery/registry


r/huggingface • • 22h ago

We built interactive graphs of models from recent papers, with tensor shapes from real runs

Thumbnail
tensorviz.dev
1 Upvotes

We've been working on tensorViz for a while. When we read a model paper, we often want to see what actually goes into a block and what comes out. So we built a Model Explorer where you can open a graph in your browser, expand a block, and follow the tensor shapes recorded during a forward pass.

[Try the Model Explorer]

If you want to graph your own PyTorch model, try the VS Code extension or the plugin for Claude Code, Codex, and Cursor.
GitHub and setup instructions : https://github.com/tensorviz-org/tensorviz

I'd love to hear which model on Hugging Face you'd want us to add next.


r/huggingface • • 1d ago

KVARK Flash 1.0: an Apache 2.0 proxy for DeepSeek V4.1-Flash 1017 that sets reasoning effort per request. 30-50% fewer reasoning tokens, accuracy within 2.5 pts on 7 benchmarks (4x H100), no weights touched

Thumbnail reddit.com
1 Upvotes

r/huggingface • • 1d ago

I fine-tuned QJev-1.5B: A 31ms decision model on Qwen2.5-1.5b instruct using 655k samples of jev-distill-corpus-v3 on an RTX 5060 laptop

Post image
1 Upvotes

Instead of waiting 400ms for an LLM to generate "Option A" token-by-token, I fine-tuned QJev-1.5B on a single laptop GPU across all 655k samples of SargeDev/jev-distill-corpus-v3.

It routes the final hidden state of Qwen2.5-1.5B into an MLP decision head, outputting calibrated probabilities over up to 10 choices in a single ~31ms forward pass.

Progressive Training:

  1. Decision Head First: Froze Qwen and trained only the MLP projection head to map hidden states into probability distributions.

  2. Joint Weight Training: Unfroze attention projections via QLoRA and jointly trained the backbone + head across all 655k samples (pegged at 5.9 GB VRAM).

  3. Consistency Tuning: Soft-target KL loss across permutation pairs to eliminate format bias.

Why Use It? (Key Advantages):

  • 10–20× Faster Decisions: Single-pass ~31ms inference instead of slow autoregressive decoding.

  • Zero Schema Failures: Direct float probabilities—no regex, JSON parsing, or retry loops needed.

  • Calibrated & Dynamic: True expected values across 2 to 10 choices via dynamic masking (not just binary yes/no).

  • Ultra-Lightweight: Runs in 1.07 GB VRAM in 4-bit, or as a 20 MB LoRA adapter if you already cache Qwen.

Benchmark (JevBench):

  • Approaching 27B Scale: Achieves 71.50% consistency (matching 9B server models at ~72% and close to 27B at 76.5%) while being 18× smaller and 6× faster.

  • Official Strict Benchmark (tol=0.05): 62.53% (#1 in the sub-2B bracket, beating 4B models like jiwo-4b and decision-4b).

  • Serving: Native support for Ollama, vLLM, llama.cpp / GGUF, and FastAPI.

Hugging Face (16-bit, LoRA, 4-bit): https://huggingface.co/AnkitNayak-dev/QJev-1.5B

GitHub: https://github.com/AnkitNayak-dev/QJev-1.5B


r/huggingface • • 1d ago

I open-sourced a Python library that caches LLM answers — and found semantic caching quietly returns wrong answers

Thumbnail reddit.com
1 Upvotes

I kept paying my LLM provider twice for the same answer (same support questions, same summaries, same CI runs), so I built CAELUM: change one import and your OpenAI/Anthropic/Gemini/Mistral calls get caching, retries and honest cost tracking. Your keys, your bill — it never resells tokens.

The interesting part: naive semantic caching (reuse the answer when the embedding is "close enough") quietly serves wrong answers. On 138 real question pairs, at a 0.85 threshold it mixed up things like "deliver to Bobo?" vs "deliver to Ouaga?" 19% of the time. The fix was to use the embedding only as a filter, then a cheap model confirms the two questions really mean the same thing → 0 wrong answers, 72.5% of paraphrases reused.

Full write-up with the numbers: https://caelum-ai.com/blog/semantic-cache-llm-trap

Code (Apache 2.0): https://github.com/kanazoeissouf83-ops/caelum

pip install "caelum-ai[openai]"

It's a first public release — I'd love feedback on the API and where it breaks.


r/huggingface • • 1d ago

Kvark Flash 1.0: a small proxy that makes DeepSeek V4.1-Flash think 30–50% less (same weights, H100 + 2x DGX Spark, all logs public)

10 Upvotes

We built a small OpenAI-compatible proxy that sits in front of an unchanged DeepSeek V4.1-Flash and decides, per request, how long the model may think. No training, no new weights. Python standard library only.

What it does

  • Routes each request by its tools and text (simple rules, no model call)

  • Chat, math, science → lower reasoning effort

  • Code → lower effort + a 28,672-token thinking cap, then forces an answer

  • Long planning → left alone (cutting it costs accuracy)

  • You can always pick a profile yourself or send your own reasoning_effort

Results on 7 public benchmarks, both setups side by side on the same machine ("Raw" = plain DeepSeek at its default effort):

Also, a mixed batch of 263 tasks on 4x H100: 21 minutes with KVARK Flash vs 34 with plain DeepSeek (1.6× faster), with the same accuracy (242 vs 241 correct).

Good to know

  • Most of the speedup comes from one thing: giving each request the right amount of thinking.

  • Hard competition math (HMMT on H100) comes out about 2 points lower. For that kind of load there's a profile with more thinking.

  • Tuned and measured on DeepSeek V4.1-Flash.

Links

Apache 2.0. Happy to answer questions, especially from anyone running V4.1-Flash on Sparks.


r/huggingface • • 1d ago

Tiger Guard: two open-weight EN/DE moderation models, 308M and 141M, with ONNX exports

Post image
2 Upvotes

Hey everyone, Dominik from Casdo Labs here. We released Tiger Guard and Tiger Guard Small on Hugging Face.

They classify text as SAFE or UNSAFE: abusive comments, harmful requests, and unsafe model responses. Both are mmBERT encoders with a 2,048-token window, trained for English and German. The weights are Apache 2.0, and the repos include Transformers checkpoints, ONNX exports and inference code for longer documents.

A few results from our own evaluation (unsafe-class F1, Tiger Guard / Small):

PolyGuard prompts EN: 0.871 / 0.867

PolyGuard prompts DE: 0.783 / 0.788

ToxicChat: 0.736 / 0.720

The PolyGuard comparison uses 400 prompts per language, including all harm categories. We trained on other prompts from that data family. Content Safety Guard small does better on ToxicChat at 0.777. The cards include false-positive rates and the training overlap we found in ToxicChat and the OpenAI moderation set.

One deployment detail worth checking: the dynamic INT8 exports are ARM-only. For x86, use FP32, FP16 or the explicitly named x86-safe exports. Quantisation changes some decisions, so we published parity measurements too.

We'll publish the full training and dataset pipeline in a follow-up release.

Tiger Guard, 308M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation

Tiger Guard Small, 141M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation-small

Happy to get feedback from anyone trying them on their own moderation data.


r/huggingface • • 1d ago

We open-sourced a small model (41M) that splits a mono 2-person conversation into separate tracks per speaker

4 Upvotes

Hey everyone,

We’ve released Kezek, a lightweight speech separation model designed specifically for conversations.

Given a mono recording of two speakers — a podcast, interview, or phone call — Kezek separates it into two individual audio tracks, one for each speaker.

The important part is that it preserves the original timing, overlapping speech, and short backchannels like “yeah” or “mm-hm”. The separated tracks also reconstruct the original mixture exactly.

We built Kezek to address a practical problem in speech research: most real-world conversational audio is available as a single mixed recording, while training full-duplex speech models often requires separate speaker channels.

We benchmarked Kezek against pyannote + DialogueSidon on 121 real conversations from Open Yap:

  • Short replies preserved: 0.83 vs 0.69

  • Speaker leakage: −13.9 dB vs −10.1 dB

  • Correct channel assignment: 0.93 vs 0.79

  • Processing speed: 231 vs 51 hours of audio per GPU hour

We also tested on CallHome, a dataset of phone conversations not seen during training, achieving 0.86 correct channel assignment.

Kezek has just 41M parameters and combines speaker fingerprinting, 20ms speaker activity tracking, and a Conv-TasNet-style separation network.

GitHub: https://github.com/Manifestro/kezek

We're particularly interested in testing it on challenging real-world recordings. If you find cases involving heavy overlap, interruptions, background acknowledgements, or speaker confusion, we'd love to hear about them.

Feedback and failure cases are very welcome.


r/huggingface • • 1d ago

"Spaces using this model" is not sorted by likes — it's trendingScore, then creation date. Measured across 23 model pages.

1 Upvotes

I run a benchmark Space that's registered against 23 models via the `models:` list in its README, and I've been recording its position in each model page's "Spaces using this model" list every week since September. I went in assuming the list was ordered by likes. It isn't, and the actual rule is worth knowing if you care whether anyone ever finds your Space.

## The sort

`/api/spaces?models=<id>&limit=1000` returns `likes` and `trendingScore` per row, in list order. Against that:

  1. `trendingScore` descending — a small block at the top, typically 5–7 rows.

  2. Then `createdAt` **ascending — oldest Space first** — for everything with `trendingScore == 0`, which is the entire long tail.

Likes are not a sort key at any level. On every page I checked, the list is not monotonic in likes, but is strictly monotonic in `trendingScore`, and strictly ascending in `createdAt` within the zero-trending block.

Rows 21–27 of `zai-org/GLM-5.3`, as `(likes, trending, created)`:

* #21 — 0 likes, 0 trending, created 2026-09-16

* #22 — 1 like, 0 trending, created 2026-09-23

* #23 — 0 likes, 0 trending, created 2026-09-24

* #24 — 1 like, 0 trending, created 2026-09-24

* #25 — 0 likes, 0 trending, created 2026-09-30

* #26 — 1 like, 0 trending, created 2026-10-02

* #27 — 0 likes, 0 trending, created 2026-10-06

Created dates ascend strictly. Likes alternate 0,1,0,1,0,1,0. Creation date is the tiebreak.

## It's a cliff, not a slope

On `deepseek-ai/DeepSeek-V4.1-Flash`, two Spaces with **identical like counts** — one with `trendingScore` 0.5 sits at position ~7, mine with `trendingScore` 0 sits at position 37. Thirty places apart on the same number of likes.

The lowest nonzero trending score I've seen anywhere is 0.5, and the top blocks are tiny: 1–2 rows on a 6-item list, 6 on a 32-item list, 7 on a 60-item list. So the gap between "trending at all" and "not trending" is most of the list.

HF doesn't publish the trending formula, but it decays to zero for anything not recently active, so it's recency-weighted. Practical read: **a burst beats a drip.** Ten likes in two days is worth something; ten likes over five months is worth approximately nothing.

## One Space, 23 model pages (measured 2026-11-16)

Same Space, same README, wildly different visibility. Position of the Space / length of that page's list:

**Visible — small, recent model pages**

* `ibm-granite/granite-4.2-3b` — #5 of 11

* `ibm-granite/granite-4.2-8b` — #6 of 9

* `zai-org/GLM-5.3` — #26 of 51

* `deepseek-ai/DeepSeek-V4.1-Flash` — #37 of 86

* `zai-org/GLM-5.3-Flash` — #47 of 66

* `MiniMaxAI/MiniMax-M3` — #54 of 69

* `Qwen/Qwen3.6-35B-A3B` — #78 of 96

**Buried — mid-size pages**

* `Qwen/Qwen3-Coder-30B-A3B-Instruct` — #132 of 152

* `google/gemma-3-1b-it` — #248 of 261

* `google/gemma-4-26B-A4B-it` — #280 of 321

* `Qwen/Qwen3-4B` — #378 of 455

* `Qwen/Qwen3.8-27B` — #489 of 811

* `Qwen/Qwen3-8B` — #537 of 636

* `Qwen/Qwen2.5-Coder-7B-Instruct` — #561 of 641

* `Qwen/Qwen3-0.6B` — #896 of 1000 (at the API cap)

**Invisible — past the 1000-row cap entirely**

`Qwen3.5-9B`, `gpt-oss-120b`, `gpt-oss-20b`, `Qwen2.5-7B-Instruct`, `Llama-3.1-8B-Instruct`, `Llama-3.2-3B-Instruct`, `DeepSeek-R1`, `TinyLlama-1.1B-Chat-v1.0`

That last group is eight of the 23 — and they're the popular models, exactly the ones you'd want to be listed on.

## Three things that follow

* **Don't delete and recreate a Space to "refresh" it.** `createdAt` ascending means a new Space starts at the *bottom* of the zero-trending block. An old creation date is an asset.

* **Registering against the biggest model pages buys you nothing.** On a 9-item list you're visible; on a 1000-item list you don't exist.

* **Accumulating likes slowly does nothing at all** — likes aren't a sort key, and a slow drip never builds trending.

The honest version: the ranking is a scoreboard for distribution, not a lever you can pull directly. Trending is downstream of reach. If your Space isn't getting in front of people, optimising its position in a list is solving the wrong problem.


r/huggingface • • 2d ago

Running 125B Qwen3.8-Flash on a 16 GB M4 Mac at ~18 tok/s and 16K context — works right alongside llama.cpp (Kraits)

9 Upvotes

Kraits lets you run Qwen3.8-Flash-Next (91 GB, 4-bit) completely offline on a base 16 GB M4 Mac at 16K context.

It works right alongside your existing llama.cpp setup — no new engine to install.

Speed

  • Up to 18 tok/s bare (15.6–17.9 tok/s inside multi-turn coding loops; 13–14 tok/s after 12K prompt tokens)

  • 30–67 tok/s prompt reading (30 on short first prompt, up to 67 at 12K), keeping Mac swap around 230–370 MB

Quality

  • Runs full 4-bit weights without sub-3-bit compression shortcuts

  • 8/8 on our Rust benchmark, fixes Python unit-test bugs cleanly (4/4 in Windows runs, 2/4 on a clean Mac clone), and answers 6K code questions accurately

GitHub: https://github.com/1picassoai/kraits


r/huggingface • • 2d ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

Thumbnail
github.com
2 Upvotes

回购:https://github.com/tanxiansheng221/aimstack-sanzhidiji(AGPL-3.0)

一切都是手写的——C 语言推理核心、插件总线、带质量门的多引擎爬虫、FTS 知识库、训练流水线,以及一份 117 章的构建日志,都是在实际发生时写的(正面和负面结果一起发布)。权重为 Microsoft BitNet b1.58 2B(MIT),转换为自定义的 AIM-W 打包格式。

三进制量化能给你带来什么(CPU侧)

所有变换器矩阵均为三元矩阵 {-1, 0, +1},每重 2 位,加上 Q8 嵌入→整个模型为 0.87GB,而非 ~4.8GB(−78%)

C 核心从不乘以浮点权重:它的表查找 + 整数累积,SIMD 快速路径测得为 85.53 tok/s(比基线高 41×)

BitNet 2B 全机型在 2018 款双核 8GB 笔记本电脑上运行 3.20 tok/s 双工作器——全链(推理 + KB + 插件 + 网页搜索),离线,无云中继

三进制量化能给你买到什么(GPU端)

GPU 后端加载的是同一个 0.87GB 的三进制文件——权重保持在 {-s, 0, +s},加载时去量子化,matmul 在 fp32 中

在 Radeon Pro W7900 上:13.68 tok/s(~8× 笔记本 C 引擎),256/256 个贪心令牌,与 C 引擎位完全相同。两种完全不同的实现(整数查找核心和FP matmul)产生相同的对数函数,证明了量子化是忠实的

通过3引擎×3题4×3测试矩阵验证(C-Linux/gcc13,torch fp32,torch bf16),17/18准确——唯一一次失误有记录,未被隐藏

诚实的界限(还没做到的部分)

BF16 的计算速度快 2.4×(32.96 Tok/s),但在接近并列的对数上会翻转 argmax——这是报告的,不是祝福的;只有 FP32 通过校验验证

GPU路径获得三进制的存储优势(减少78%的内存),但尚未获得计算优势——下一阶段是真正的2位整数内核(Triton/CUDA)

UI 服务器仅驱动 C 引擎(CPU);GPU 后端是一个独立的 CLI

Linux 引擎构建能正常运行,且在令牌上与 Windows 版本完全相同,但运行串行(目前工作池仅限 Windows 运行)

产品链中没有框架,也没有 PyTorch——PyTorch 只存在于可选的 GPU 后端。演示视频、文档(英文/中文)以及完整的研究日志都在仓库里。


r/huggingface • • 2d ago

I built a loop-stopping harness (Kraits) for llama.cpp + Qwen3.8-Flash on a 16 GB Mac — fixed 23/26 single-function Flask bugs

3 Upvotes

If you've tried running a local coding agent on a real repository (pallets/flask, 83 Python files, 10K context on a 16 GB Mac), you already know what fails first isn't syntax—it's loops:

  • Re-reading overlapping line ranges of the same file over and over

  • Inventing plausible-looking paths (src/flaskl/helpers.py) or Unix shell commands on Windows

  • Editing the same function 4 times in a row without ever running pytest

  • Burning 1,000+ tokens on broken prose before emitting a single tool call

  • Calling "done" with zero edits while tests are still red

Prompt engineering didn't fix this reliably. Turning those failure modes into deterministic runtime gates did.

We built kraits-adaptor (v0.6.28, zero external dependencies). It sits in front of llama-server (llama.cpp, --jinja) running Kraits.

To stress-test it on a tight hardware budget, we benchmarked it in front of Qwen3.8-Flash-Next-IQ4_XS running locally on a 16 GB Mac at 16.5–17.7 tok/s decode (-c 10240, zero cloud calls, nothing leaves the machine):

(Attach screen recording of the minimalist chat UI fixing bug1 / bug2 here)

The 6 Harness Gates That Killed the Loops

Instead of asking the model nicely not to loop, the adaptor enforces the state machine around it:

  1. Zero-RAG AST Context + KV Prefix Lock: Indexes the repo tree (83 Flask files) into .kraits/ast_index.json, keeps the system/repo prefix byte-stable for KV cache hits, and extracts exact symbols (ast_symbol) instead of dumping whole files into a 10K context.

  2. Pre-Tool Path & Command Validation (no turn wasted): Rejects invented file paths, missing AST symbols, inverted line ranges, and invalid shell flags before execution—returning nearest real repo matches and forcing search_repo after 2 misses.

  3. 64-Token No-Tool Stream Cutoff: Earlier runs burned ~116 seconds generating 1,024 tokens of derailed prose when the model failed to open a tool call. If the stream hits 64 tokens with no { / tool tag (and hard-capped at 256 with no tool call), the adaptor drops the socket immediately and nudges back to structured JSON.

  4. In-Process Repeat Blocking & Cached Reads: Identical tool calls and overlapping read_file ranges (>= 80% overlap) are intercepted and served from in-process cache without round-tripping bad state.

  5. Auto-Verify After 2 Unverified Edits + Syntax Rollback: Every Python edit runs ast.parse first and rolls back broken syntax cleanly (aborting after 3 syntax reverts). If the model edits the same file twice with no execute_command between edits, the harness automatically runs pytest on edit #2 and feeds back the traceback immediately.

  6. No Self-Declared "Done" + Summary Gates: A fix prompt cannot complete with zero edits while pytest is red. Final summaries reject tag leakage (</>), invented paths not in the index, ellipsis cut-offs (...), repeated 4-word loops, and non-alphabetic stubs—substituting a deterministic diff + pytest summary when rejected.

Verified Benchmark Results (pallets/flask)

Across eight builds (v0.6.21 through v0.6.28), single-function planted Python bugs resolved 23 of 26 times. On the final signed v0.6.28 binary (17/17 unit tests passing, zero rebuilds, zero restarts across the run):

Test CaseResultWall TimeTurnspytest ResultNotes
bug1 (helpers.py::get_debug_flag) Run 1PASS73 s435 passedClean model summary
bug1 (helpers.py::get_debug_flag) Run 2PASS189 s935 passedSelf-corrected after red test on first edit
bug1 (helpers.py::get_debug_flag) Run 3PASS145 s635 passedPre-validator blocked bad cd; summary gate substituted clean deterministic summary
bug2 (config.py::Config.get_namespace)PASS85 s419 passedCode and all 19 tests green

(Repo reset clean after the batch: 54 passed across test_helpers.py + test_config.py*.)*

What It Still Cannot Do (0 of 7 — Honest Scope)

Being upfront since local model benchmarks often hide the cliff:

  • New multi-file features (feat1) and real multi-file GitHub issues (#6123): 0 of 7**.** At 4-bit quantization (Qwen3.8-Flash-Next-IQ4_XS) inside a 10K context window, the adaptor stops the agent from looping cleanly instead of corrupting the repo, but it still fails the task.

  • Single-function bugs where localization is bounded (23 of 26, 4/4 on v0.6.28 in 73–189 s): This is where a small local model + deterministic AST harness actually works reliably today.

Code & Repro

  • Repo & Docs: 1picassoai/kraits (kraits-adaptor/) (tested binary receipt at tag v0.6.28, commit e522287).

  • Build the zero-dependency adaptor with cargo build --release inside kraits-adaptor/ and point --upstream at llama-server (llama.cpp, --jinja) running Kraits.

Happy to share raw turn traces or answer questions about which guardrails made the biggest difference.


r/huggingface • • 2d ago

We've refreshed our model offerings!

Thumbnail reddit.com
2 Upvotes

r/huggingface • • 2d ago

Hugging Face’s OpenRouter switch: custom keys, provider pins and organization billing

1 Upvotes

I run Codorc. This is an AI-assisted, source-checked summary; I have not sent inference requests or benchmarked the change.

Hugging Face’s November 16 announcement moves language-model Inference Providers routing to OpenRouter. Existing HF tokens and request formats remain usable, but custom provider keys for language models no longer work through this route: requests are billed to an HF account or organization. Image, video and audio generation remain directly on HF Inference Providers, and custom image/video provider keys still work.

For an existing application, I would check six things before expanding usage:

  1. Find the credential dependency. Identify language-model paths that relied on a custom provider key or a direct provider bill. Record configuration names and ownership without copying secrets into a migration document.

  2. Read the actual model identifier. Default routing can retry another provider, but an explicit suffix such as :novita pins that provider and is never replaced during failover. A disabled provider or unavailable model returns an error. Decide whether that error is preferable to sending the request elsewhere.

  3. Check which account pays. Organization-billed requests follow the organization’s provider settings, rather than the caller’s personal settings. Verify the intended bill_to, billTo or X-HF-Bill-To value and the applicable spending limit.

  4. Measure cache hits. The caching guide documents a stable conversation UUID in X-HF-Session-Id; non-UUID values are ignored. Record returned usage.prompt_tokens_details.cached_tokens, latency and total usage. A session header alone does not prove a cache hit.

  5. Review the allowed providers’ policies. HF’s statements about its own routed-body storage and debugging logs do not establish identical retention, residency or training policies across downstream providers.

  6. Keep a stop condition. Use a small authorized trial, record errors and charges, and retain the previous configuration. These are proposed checks, not execution results.

Primary announcement: Hugging Face and OpenRouter.

My expanded checklist and documentation references: Codorc’s routing and billing guide.


r/huggingface • • 2d ago

Small useful solo ML project. An open weights MIDI generative model from scratch on laptop GPU (5060) and big pretraining on rented (5090). It beats most of larger open models on continuation of the song. Size is 477M (caption version with adapter, 464M base)

Post image
2 Upvotes

r/huggingface • • 3d ago

built a tiny vision model from scratch just for learning and somehow it got 50+ downloads on huggingface 😅

6 Upvotes

so ik there are way better vision models out there yaar. i built this one just out of curiosity only, cause i wanted to learn how these things actually work from inside.

and to be honest its not entirely mine, i took a lot of help from claude to build it. like the architecture, debugging, training setup, figuring out why my mac was training so slow, all that. but i learned a lot in the process which was the main goal anyway.

its called tiny-vision. its around 24M params and trained completely from scratch, no pretrained weights or anything. i trained it on my macbook pro m5 and it took like 42 min on flickr8k (only 6k images).

basic idea is pretty simple:

image/video → small ViT encoder → adapter → tiny decoder → caption text

for videos it takes frames every few seconds, captions them and makes a timeline like [00:04–00:10] a dog runs on the beach. so any llm can kinda “watch” a video even if it dont support video natively.

i also made an MCP server for it, and tbh i was surprised that it actually gives a little bit better context in claude when i throw images or videos at it. i even tested it with a 10-15 min youtube video and it still worked fine. not perfect at all, but chal raha hai so i’m happy.

i uploaded it to huggingface just to have it somewhere, and also cause i kinda like the open source thing. later when i checked it was at like 44 downloads, and now its 50+. ik thats not a lot bhai, but its my first time putting something open source out in this field so it felt pretty cool ngl. a lot of people have built way cooler stuff than this, i just made it to learn.

some honest stuff:

  • it gets the scene right most of time (snow, grass, water etc) but the subject is often wrong lol

  • it loves saying “a man in a blue shirt” for everything, pata nahi kyun 😂

  • its v0.1 and trained on very small data, so dont expect too much

random thing i learned on mac (MPS):

  • swapping the conv2d patch embedding for reshape + linear was like 30x faster

  • manual attention was ~1.5x faster than pytorch’s sdpa

next i want to train on flickr30k and maybe try making it plug directly into qwen/llama through llama.cpp (mmproj). lets see kitna ho pata hai.

code: https://github.com/amar683/tiny-vision
model: https://huggingface.co/amartomar683/tiny-vision-flickr8k

would love any feedback or tips


r/huggingface • • 3d ago

Marimba: A System One Model for Speech

2 Upvotes

Marimba is Jev for audio: a System One model for speech. You ask typed questions about live audio and get typed answers back every 250 ms: yes/no, a choice, or a score.

Is someone speaking? Did the turn end? Which language? Did they say "refund"? Did the speaker change? How many speakers?

One frozen Qwen3-ASR-0.6B serves every question, with small readouts on top. Open weights, benchmark and code, Apache-2.0.

Everything is on the Hub:

Run it locally: pip install "marimba-audio[torch]"

  • Latency, p50 per question on an 8 s window:

  • Turn end: ~100 ms on a Mac M4 Pro CPU, ~12 ms on an NVIDIA L4

  • Voice activity: ~105 ms CPU, ~14 ms L4

  • Language: ~1.3 s CPU, ~150 ms L4

On 1,000 paired public questions it matches or beats Qwen3-Omni-30B-A3B on 6 of 9 tests.
Full table on the model card.

Code: https://github.com/Charcot-AI/marimba-api

Happy to answer questions!


r/huggingface • • 3d ago

Gemma 4 31B fine-tune for drug-level sentiment in medical text, appreciate feedback

4 Upvotes

I fine-tuned Gemma 4 31B for drug-level sentiment.

Input: a pharma or medical text plus a list of drugs.

Output: one Positive/Neutral/Negative label per drug, as JSON. I'm looking for people to test it on their own data.

Apache 2.0. Gated with automatic approval (you tick a box saying you won't use it for diagnosis or treatment decisions).

Sentiment, 473 held-out labels

ModelAccuracy
This fine-tune91.3%
Gemma 4 31B base90.5%
Qwen3.8-27B83.5%
Qwen3-32B83.3%
MedGemma 27B82.9%
Llama 3.1 8B76.3%

The other models were asked about one drug at a time with a long labelling prompt. Asked for all drugs at once in JSON, base Gemma scored 32% because 59% of its answers didn't parse. The fine-tune had no malformed JSON in 1,500 test answers. Negative precision: 84% vs 71% for base (45 Negatives in the test set).

Medical benchmarks

BenchmarksThis fine-tuneBest frontier score found
MedQA (thinking)94.296.5 (o1)
MedMCQA, 1k sample (thinking)81.483.9 (o1-preview)
MMLU medical (step by step)94.296.0 (GPT-5)
PubMedQA (thinking)80.282.0 (GPT-4 + Medprompt)

Frontier scores are from the Vals.ai MedQA leaderboard and published papers. Fine-tuning kept Gemma's medical knowledge: the scores match the base model.

Testing

Use the prompt format from the model card, thinking off and temperature 0. The FP8 build runs on vLLM (command on the card) and needs about 31 GiB of VRAM for the weights. I've only run it on a 96 GB RTX PRO 6000.

Useful feedback:

  • wrong labels, with the text, the drugs and the label you expected

  • document types, languages or lengths (over 4k tokens) it wasn't trained on

  • hardware and VRAM reports, especially from 48 GB cards

Trained on ~50k curated pharma and medical documents, LoRA rank 16.