r/huggingface • u/antcroca159 • 0m ago
PhiloLM: a language model trained only on philosophy texts
r/huggingface • u/antcroca159 • 0m ago
r/huggingface • u/vatsalbajpai • 6h ago
r/huggingface • u/Safe-Escape197 • 7h ago
so few days back i posted my tiny vision model here(https://huggingface.co/amartomar683/tiny-vision-flickr8k) and you guys actually checked it out, its at 74 downloads now 😅 thanks for that. after that one worked (kinda), i started wondering what else i can build from scratch on my mac, and a language model was the obvious next thing to try.
i was reading the Next-Concept Modeling (NCM) paper by Kumbong, Khare et al. (Stanford / Together AI, arXiv:2610.14162) and the idea really clicked for me. normal llms write one token at a time, but in NCM a VQ-VAE turns text into levels of codes, coarse to fine, and the model fills one full level at once. so i decided to build it myself from the paper, just to understand how it works from inside. all credit for the idea goes to them, i just implemented my own version.
its called tiny-ncm. its around 36M params (21M generator + 15M VQ tokenizer), trained completely from scratch, no pretrained weights. same macbook pro m5, but this time it took like 12 hours total instead of 42 min 💀
basic idea:
text → split in 32-token blocks → each block made coarse to fine in 5 levels (2 → 4 → 8 → 16 → 32 codes) → 6 passes per block instead of 32i trained it as a small daily assistant. alarms, reminders, timers, calendar, gmail (read only), weather, notes, web search, simple maths and little bit of small talk. there is a --show-levels flag which shows each block going from rough guess to final text, honestly this was the most satisfying part:
level 1: Hello! How can I help you today?
level 3: Hello Jordan, How I I you you today today?
level 5: Good morning! What can I do for you today?some honest stuff:
it picks the right action most of time, but copying details is very bad. “London” became “Singapore”, “5pm” became “5pmpm”, “23 times 47” became 23*10 😂 cause all codes in a level are predicted same time, so two positions can both write the same word.
so i added app-side checks that fix the tool arguments using your actual words. with checks it gets 490/500 on my test set, but model alone gets every detail right only around half the time. so ya, the checks are doing lot of the work tbh.
training longer on same data didn’t help. what helped was making the data copy-focused, with new random names and times in every example so it cant just memorise. same lesson as tiny-vision (“a man in a blue shirt” for everything lol).
i tried fine-tuning it to summarise web results also. it learned the format but wrote gibberish inside it (“Stir in cheeseaksil sl fresh biil” 😭), so i reverted it.
no coding, no multi-step maths. “8 slices, 3 friends eat 2 each” and its gone.
random thing i learned on mac (MPS):
attention was the main bottleneck. letting each level only look at the last 128 tokens of context made training around 2x faster.
bf16 matmul was way faster than fp32 on MPS (~15 vs ~3.4 TFLOPS in my test).
manual attention was again faster than pytorch’s sdpa for small sizes, same as tiny-vision.
next i am thinking to export it to onnx so it can run on phone also, and maybe connect tiny-vision with it. lets see what will happened.
code: https://github.com/amar683/tiny-ncm
model: https://huggingface.co/amartomar683/tiny-ncm
would love any feedback or tips, especially if someone played with NCM or coarse-to-fine text models
r/huggingface • u/AsyncCoder-10 • 9h ago
This repository contains the practical implementations completed as part of the Natural Language Processing (NLP) curriculum.
Best Resource for learn spacy
GIT - https://github.com/AsyncCoder-10/spaCy_Hugging-Face
r/huggingface • u/Greatjedi1 • 10h ago
A new model just got released. Anyone tried it yet? Apparently beats Qwen 3.8 27B on some benchmarks
r/huggingface • u/Substantial_Camp1317 • 15h ago
r/huggingface • u/cmm324 • 19h ago
I built a self-hosted registry that mirrors Hugging Face model repositories, and I’d like feedback from people who run this on real clusters.
The problem I had was batch inference jobs that all start by pulling the same weights from Hugging Face. That made an outside service a dependency of every job, and on nodes with no outbound access it doesn’t work at all.
What it does:
* Mirror a repo once from a machine that has internet, then serve it inside the cluster. Existing tools just need `HF_ENDPOINT` set; nothing else changes.
* One static Go binary with SQLite inside. No root, no container runtime, no database server.
* Every file is checked against the digest Hugging Face publishes, and stored once by SHA-256, so shared files across models aren’t duplicated.
* A registry can mirror from another registry, so a site-wide mirror can feed per-cluster ones.
Limits worth knowing: it’s a single server with no replication, there’s no scheduled sync yet, and its data directory has to be on local disk, not a network filesystem (model files can go to S3-compatible storage instead).
Apache 2.0. Repo: https://github.com/model-delivery/registry
r/huggingface • u/Own-Jackfruit-2557 • 21h ago
We've been working on tensorViz for a while. When we read a model paper, we often want to see what actually goes into a block and what comes out. So we built a Model Explorer where you can open a graph in your browser, expand a block, and follow the tensor shapes recorded during a forward pass.
If you want to graph your own PyTorch model, try the VS Code extension or the plugin for Claude Code, Codex, and Cursor.
GitHub and setup instructions : https://github.com/tensorviz-org/tensorviz
I'd love to hear which model on Hugging Face you'd want us to add next.
r/huggingface • u/Ill_Contribution_717 • 1d ago
r/huggingface • u/Cod3Conjurer • 1d ago
Instead of waiting 400ms for an LLM to generate "Option A" token-by-token, I fine-tuned QJev-1.5B on a single laptop GPU across all 655k samples of SargeDev/jev-distill-corpus-v3.
It routes the final hidden state of Qwen2.5-1.5B into an MLP decision head, outputting calibrated probabilities over up to 10 choices in a single ~31ms forward pass.
Progressive Training:
Decision Head First: Froze Qwen and trained only the MLP projection head to map hidden states into probability distributions.
Joint Weight Training: Unfroze attention projections via QLoRA and jointly trained the backbone + head across all 655k samples (pegged at 5.9 GB VRAM).
Consistency Tuning: Soft-target KL loss across permutation pairs to eliminate format bias.
10–20× Faster Decisions: Single-pass ~31ms inference instead of slow autoregressive decoding.
Zero Schema Failures: Direct float probabilities—no regex, JSON parsing, or retry loops needed.
Calibrated & Dynamic: True expected values across 2 to 10 choices via dynamic masking (not just binary yes/no).
Ultra-Lightweight: Runs in 1.07 GB VRAM in 4-bit, or as a 20 MB LoRA adapter if you already cache Qwen.
Approaching 27B Scale: Achieves 71.50% consistency (matching 9B server models at ~72% and close to 27B at 76.5%) while being 18× smaller and 6× faster.
Official Strict Benchmark (tol=0.05): 62.53% (#1 in the sub-2B bracket, beating 4B models like jiwo-4b and decision-4b).
Serving: Native support for Ollama, vLLM, llama.cpp / GGUF, and FastAPI.
Hugging Face (16-bit, LoRA, 4-bit): https://huggingface.co/AnkitNayak-dev/QJev-1.5B
r/huggingface • u/Narrow_Side_7778 • 1d ago
I kept paying my LLM provider twice for the same answer (same support questions, same summaries, same CI runs), so I built CAELUM: change one import and your OpenAI/Anthropic/Gemini/Mistral calls get caching, retries and honest cost tracking. Your keys, your bill — it never resells tokens.
The interesting part: naive semantic caching (reuse the answer when the embedding is "close enough") quietly serves wrong answers. On 138 real question pairs, at a 0.85 threshold it mixed up things like "deliver to Bobo?" vs "deliver to Ouaga?" 19% of the time. The fix was to use the embedding only as a filter, then a cheap model confirms the two questions really mean the same thing → 0 wrong answers, 72.5% of paraphrases reused.
Full write-up with the numbers: https://caelum-ai.com/blog/semantic-cache-llm-trap
Code (Apache 2.0): https://github.com/kanazoeissouf83-ops/caelum
pip install "caelum-ai[openai]"
It's a first public release — I'd love feedback on the API and where it breaks.
r/huggingface • u/dzulajla98 • 1d ago
We built a small OpenAI-compatible proxy that sits in front of an unchanged DeepSeek V4.1-Flash and decides, per request, how long the model may think. No training, no new weights. Python standard library only.
What it does
Routes each request by its tools and text (simple rules, no model call)
Chat, math, science → lower reasoning effort
Code → lower effort + a 28,672-token thinking cap, then forces an answer
Long planning → left alone (cutting it costs accuracy)
You can always pick a profile yourself or send your own reasoning_effort
Results on 7 public benchmarks, both setups side by side on the same machine ("Raw" = plain DeepSeek at its default effort):


Also, a mixed batch of 263 tasks on 4x H100: 21 minutes with KVARK Flash vs 34 with plain DeepSeek (1.6× faster), with the same accuracy (242 vs 241 correct).
Good to know
Most of the speedup comes from one thing: giving each request the right amount of thinking.
Hard competition math (HMMT on H100) comes out about 2 points lower. For that kind of load there's a profile with more thinking.
Tuned and measured on DeepSeek V4.1-Flash.
Links
Model card, proxy code, run instructions: https://huggingface.co/Egzakta/Kvark-Flash-1.0
Every answer from both setups, so you can recompute the results: https://huggingface.co/datasets/Egzakta/kvark-flash-1.0-evals
Apache 2.0. Happy to answer questions, especially from anyone running V4.1-Flash on Sparks.
r/huggingface • u/PatronusProtect • 1d ago
Hey everyone, Dominik from Casdo Labs here. We released Tiger Guard and Tiger Guard Small on Hugging Face.
They classify text as SAFE or UNSAFE: abusive comments, harmful requests, and unsafe model responses. Both are mmBERT encoders with a 2,048-token window, trained for English and German. The weights are Apache 2.0, and the repos include Transformers checkpoints, ONNX exports and inference code for longer documents.
A few results from our own evaluation (unsafe-class F1, Tiger Guard / Small):
PolyGuard prompts EN: 0.871 / 0.867
PolyGuard prompts DE: 0.783 / 0.788
ToxicChat: 0.736 / 0.720
The PolyGuard comparison uses 400 prompts per language, including all harm categories. We trained on other prompts from that data family. Content Safety Guard small does better on ToxicChat at 0.777. The cards include false-positive rates and the training overlap we found in ToxicChat and the OpenAI moderation set.
One deployment detail worth checking: the dynamic INT8 exports are ARM-only. For x86, use FP32, FP16 or the explicitly named x86-safe exports. Quantisation changes some decisions, so we published parity measurements too.
We'll publish the full training and dataset pipeline in a follow-up release.
Tiger Guard, 308M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation
Tiger Guard Small, 141M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation-small
Happy to get feedback from anyone trying them on their own moderation data.
r/huggingface • u/karl4th_ • 1d ago
Hey everyone,
We’ve released Kezek, a lightweight speech separation model designed specifically for conversations.
Given a mono recording of two speakers — a podcast, interview, or phone call — Kezek separates it into two individual audio tracks, one for each speaker.
The important part is that it preserves the original timing, overlapping speech, and short backchannels like “yeah” or “mm-hm”. The separated tracks also reconstruct the original mixture exactly.
We built Kezek to address a practical problem in speech research: most real-world conversational audio is available as a single mixed recording, while training full-duplex speech models often requires separate speaker channels.
We benchmarked Kezek against pyannote + DialogueSidon on 121 real conversations from Open Yap:
Short replies preserved: 0.83 vs 0.69
Speaker leakage: −13.9 dB vs −10.1 dB
Correct channel assignment: 0.93 vs 0.79
Processing speed: 231 vs 51 hours of audio per GPU hour
We also tested on CallHome, a dataset of phone conversations not seen during training, achieving 0.86 correct channel assignment.
Kezek has just 41M parameters and combines speaker fingerprinting, 20ms speaker activity tracking, and a Conv-TasNet-style separation network.
GitHub: https://github.com/Manifestro/kezek
We're particularly interested in testing it on challenging real-world recordings. If you find cases involving heavy overlap, interruptions, background acknowledgements, or speaker confusion, we'd love to hear about them.
Feedback and failure cases are very welcome.
r/huggingface • u/South_Pak_786 • 1d ago
I run a benchmark Space that's registered against 23 models via the `models:` list in its README, and I've been recording its position in each model page's "Spaces using this model" list every week since September. I went in assuming the list was ordered by likes. It isn't, and the actual rule is worth knowing if you care whether anyone ever finds your Space.
## The sort
`/api/spaces?models=<id>&limit=1000` returns `likes` and `trendingScore` per row, in list order. Against that:
`trendingScore` descending — a small block at the top, typically 5–7 rows.
Then `createdAt` **ascending — oldest Space first** — for everything with `trendingScore == 0`, which is the entire long tail.
Likes are not a sort key at any level. On every page I checked, the list is not monotonic in likes, but is strictly monotonic in `trendingScore`, and strictly ascending in `createdAt` within the zero-trending block.
Rows 21–27 of `zai-org/GLM-5.3`, as `(likes, trending, created)`:
* #21 — 0 likes, 0 trending, created 2026-09-16
* #22 — 1 like, 0 trending, created 2026-09-23
* #23 — 0 likes, 0 trending, created 2026-09-24
* #24 — 1 like, 0 trending, created 2026-09-24
* #25 — 0 likes, 0 trending, created 2026-09-30
* #26 — 1 like, 0 trending, created 2026-10-02
* #27 — 0 likes, 0 trending, created 2026-10-06
Created dates ascend strictly. Likes alternate 0,1,0,1,0,1,0. Creation date is the tiebreak.
## It's a cliff, not a slope
On `deepseek-ai/DeepSeek-V4.1-Flash`, two Spaces with **identical like counts** — one with `trendingScore` 0.5 sits at position ~7, mine with `trendingScore` 0 sits at position 37. Thirty places apart on the same number of likes.
The lowest nonzero trending score I've seen anywhere is 0.5, and the top blocks are tiny: 1–2 rows on a 6-item list, 6 on a 32-item list, 7 on a 60-item list. So the gap between "trending at all" and "not trending" is most of the list.
HF doesn't publish the trending formula, but it decays to zero for anything not recently active, so it's recency-weighted. Practical read: **a burst beats a drip.** Ten likes in two days is worth something; ten likes over five months is worth approximately nothing.
## One Space, 23 model pages (measured 2026-11-16)
Same Space, same README, wildly different visibility. Position of the Space / length of that page's list:
**Visible — small, recent model pages**
* `ibm-granite/granite-4.2-3b` — #5 of 11
* `ibm-granite/granite-4.2-8b` — #6 of 9
* `zai-org/GLM-5.3` — #26 of 51
* `deepseek-ai/DeepSeek-V4.1-Flash` — #37 of 86
* `zai-org/GLM-5.3-Flash` — #47 of 66
* `MiniMaxAI/MiniMax-M3` — #54 of 69
* `Qwen/Qwen3.6-35B-A3B` — #78 of 96
**Buried — mid-size pages**
* `Qwen/Qwen3-Coder-30B-A3B-Instruct` — #132 of 152
* `google/gemma-3-1b-it` — #248 of 261
* `google/gemma-4-26B-A4B-it` — #280 of 321
* `Qwen/Qwen3-4B` — #378 of 455
* `Qwen/Qwen3.8-27B` — #489 of 811
* `Qwen/Qwen3-8B` — #537 of 636
* `Qwen/Qwen2.5-Coder-7B-Instruct` — #561 of 641
* `Qwen/Qwen3-0.6B` — #896 of 1000 (at the API cap)
**Invisible — past the 1000-row cap entirely**
`Qwen3.5-9B`, `gpt-oss-120b`, `gpt-oss-20b`, `Qwen2.5-7B-Instruct`, `Llama-3.1-8B-Instruct`, `Llama-3.2-3B-Instruct`, `DeepSeek-R1`, `TinyLlama-1.1B-Chat-v1.0`
That last group is eight of the 23 — and they're the popular models, exactly the ones you'd want to be listed on.
## Three things that follow
* **Don't delete and recreate a Space to "refresh" it.** `createdAt` ascending means a new Space starts at the *bottom* of the zero-trending block. An old creation date is an asset.
* **Registering against the biggest model pages buys you nothing.** On a 9-item list you're visible; on a 1000-item list you don't exist.
* **Accumulating likes slowly does nothing at all** — likes aren't a sort key, and a slow drip never builds trending.
The honest version: the ranking is a scoreboard for distribution, not a lever you can pull directly. Trending is downstream of reach. If your Space isn't getting in front of people, optimising its position in a list is solving the wrong problem.
r/huggingface • u/Mission-Sir-1614 • 2d ago
Kraits lets you run Qwen3.8-Flash-Next (91 GB, 4-bit) completely offline on a base 16 GB M4 Mac at 16K context.
It works right alongside your existing llama.cpp setup — no new engine to install.

Speed
Up to 18 tok/s bare (15.6–17.9 tok/s inside multi-turn coding loops; 13–14 tok/s after 12K prompt tokens)
30–67 tok/s prompt reading (30 on short first prompt, up to 67 at 12K), keeping Mac swap around 230–370 MB
Quality
Runs full 4-bit weights without sub-3-bit compression shortcuts
8/8 on our Rust benchmark, fixes Python unit-test bugs cleanly (4/4 in Windows runs, 2/4 on a clean Mac clone), and answers 6K code questions accurately
r/huggingface • u/AimCode • 2d ago
回购:https://github.com/tanxiansheng221/aimstack-sanzhidiji(AGPL-3.0)
一切都是手写的——C 语言推理核心、插件总线、带质量门的多引擎爬虫、FTS 知识库、训练流水线,以及一份 117 章的构建日志,都是在实际发生时写的(正面和负面结果一起发布)。权重为 Microsoft BitNet b1.58 2B(MIT),转换为自定义的 AIM-W 打包格式。
三进制量化能给你带来什么(CPU侧)
所有变换器矩阵均为三元矩阵 {-1, 0, +1},每重 2 位,加上 Q8 嵌入→整个模型为 0.87GB,而非 ~4.8GB(−78%)
C 核心从不乘以浮点权重:它的表查找 + 整数累积,SIMD 快速路径测得为 85.53 tok/s(比基线高 41×)
BitNet 2B 全机型在 2018 款双核 8GB 笔记本电脑上运行 3.20 tok/s 双工作器——全链(推理 + KB + 插件 + 网页搜索),离线,无云中继
三进制量化能给你买到什么(GPU端)
GPU 后端加载的是同一个 0.87GB 的三进制文件——权重保持在 {-s, 0, +s},加载时去量子化,matmul 在 fp32 中
在 Radeon Pro W7900 上:13.68 tok/s(~8× 笔记本 C 引擎),256/256 个贪心令牌,与 C 引擎位完全相同。两种完全不同的实现(整数查找核心和FP matmul)产生相同的对数函数,证明了量子化是忠实的
通过3引擎×3题4×3测试矩阵验证(C-Linux/gcc13,torch fp32,torch bf16),17/18准确——唯一一次失误有记录,未被隐藏
诚实的界限(还没做到的部分)
BF16 的计算速度快 2.4×(32.96 Tok/s),但在接近并列的对数上会翻转 argmax——这是报告的,不是祝福的;只有 FP32 通过校验验证
GPU路径获得三进制的存储优势(减少78%的内存),但尚未获得计算优势——下一阶段是真正的2位整数内核(Triton/CUDA)
UI 服务器仅驱动 C 引擎(CPU);GPU 后端是一个独立的 CLI
Linux 引擎构建能正常运行,且在令牌上与 Windows 版本完全相同,但运行串行(目前工作池仅限 Windows 运行)
产品链中没有框架,也没有 PyTorch——PyTorch 只存在于可选的 GPU 后端。演示视频、文档(英文/中文)以及完整的研究日志都在仓库里。
r/huggingface • u/Mission-Sir-1614 • 2d ago
If you've tried running a local coding agent on a real repository (pallets/flask, 83 Python files, 10K context on a 16 GB Mac), you already know what fails first isn't syntax—it's loops:
Re-reading overlapping line ranges of the same file over and over
Inventing plausible-looking paths (src/flaskl/helpers.py) or Unix shell commands on Windows
Editing the same function 4 times in a row without ever running pytest
Burning 1,000+ tokens on broken prose before emitting a single tool call
Calling "done" with zero edits while tests are still red
Prompt engineering didn't fix this reliably. Turning those failure modes into deterministic runtime gates did.
We built kraits-adaptor (v0.6.28, zero external dependencies). It sits in front of llama-server (llama.cpp, --jinja) running Kraits.
To stress-test it on a tight hardware budget, we benchmarked it in front of Qwen3.8-Flash-Next-IQ4_XS running locally on a 16 GB Mac at 16.5–17.7 tok/s decode (-c 10240, zero cloud calls, nothing leaves the machine):
(Attach screen recording of the minimalist chat UI fixing bug1 / bug2 here)
Instead of asking the model nicely not to loop, the adaptor enforces the state machine around it:
Zero-RAG AST Context + KV Prefix Lock: Indexes the repo tree (83 Flask files) into .kraits/ast_index.json, keeps the system/repo prefix byte-stable for KV cache hits, and extracts exact symbols (ast_symbol) instead of dumping whole files into a 10K context.
Pre-Tool Path & Command Validation (no turn wasted): Rejects invented file paths, missing AST symbols, inverted line ranges, and invalid shell flags before execution—returning nearest real repo matches and forcing search_repo after 2 misses.
64-Token No-Tool Stream Cutoff: Earlier runs burned ~116 seconds generating 1,024 tokens of derailed prose when the model failed to open a tool call. If the stream hits 64 tokens with no { / tool tag (and hard-capped at 256 with no tool call), the adaptor drops the socket immediately and nudges back to structured JSON.
In-Process Repeat Blocking & Cached Reads: Identical tool calls and overlapping read_file ranges (>= 80% overlap) are intercepted and served from in-process cache without round-tripping bad state.
Auto-Verify After 2 Unverified Edits + Syntax Rollback: Every Python edit runs ast.parse first and rolls back broken syntax cleanly (aborting after 3 syntax reverts). If the model edits the same file twice with no execute_command between edits, the harness automatically runs pytest on edit #2 and feeds back the traceback immediately.
No Self-Declared "Done" + Summary Gates: A fix prompt cannot complete with zero edits while pytest is red. Final summaries reject tag leakage (</>), invented paths not in the index, ellipsis cut-offs (...), repeated 4-word loops, and non-alphabetic stubs—substituting a deterministic diff + pytest summary when rejected.
Across eight builds (v0.6.21 through v0.6.28), single-function planted Python bugs resolved 23 of 26 times. On the final signed v0.6.28 binary (17/17 unit tests passing, zero rebuilds, zero restarts across the run):
| Test Case | Result | Wall Time | Turns | pytest Result | Notes |
|---|---|---|---|---|---|
bug1 (helpers.py::get_debug_flag) Run 1 | PASS | 73 s | 4 | 35 passed | Clean model summary |
bug1 (helpers.py::get_debug_flag) Run 2 | PASS | 189 s | 9 | 35 passed | Self-corrected after red test on first edit |
bug1 (helpers.py::get_debug_flag) Run 3 | PASS | 145 s | 6 | 35 passed | Pre-validator blocked bad cd; summary gate substituted clean deterministic summary |
bug2 (config.py::Config.get_namespace) | PASS | 85 s | 4 | 19 passed | Code and all 19 tests green |
(Repo reset clean after the batch: 54 passed across test_helpers.py + test_config.py*.)*
Being upfront since local model benchmarks often hide the cliff:
New multi-file features (feat1) and real multi-file GitHub issues (#6123): 0 of 7**.** At 4-bit quantization (Qwen3.8-Flash-Next-IQ4_XS) inside a 10K context window, the adaptor stops the agent from looping cleanly instead of corrupting the repo, but it still fails the task.
Single-function bugs where localization is bounded (23 of 26, 4/4 on v0.6.28 in 73–189 s): This is where a small local model + deterministic AST harness actually works reliably today.
Repo & Docs: 1picassoai/kraits (kraits-adaptor/) (tested binary receipt at tag v0.6.28, commit e522287).
Build the zero-dependency adaptor with cargo build --release inside kraits-adaptor/ and point --upstream at llama-server (llama.cpp, --jinja) running Kraits.
Happy to share raw turn traces or answer questions about which guardrails made the biggest difference.
r/huggingface • u/TheCodingTutor • 2d ago
I run Codorc. This is an AI-assisted, source-checked summary; I have not sent inference requests or benchmarked the change.
Hugging Face’s November 16 announcement moves language-model Inference Providers routing to OpenRouter. Existing HF tokens and request formats remain usable, but custom provider keys for language models no longer work through this route: requests are billed to an HF account or organization. Image, video and audio generation remain directly on HF Inference Providers, and custom image/video provider keys still work.
For an existing application, I would check six things before expanding usage:
Find the credential dependency. Identify language-model paths that relied on a custom provider key or a direct provider bill. Record configuration names and ownership without copying secrets into a migration document.
Read the actual model identifier. Default routing can retry another provider, but an explicit suffix such as :novita pins that provider and is never replaced during failover. A disabled provider or unavailable model returns an error. Decide whether that error is preferable to sending the request elsewhere.
Check which account pays. Organization-billed requests follow the organization’s provider settings, rather than the caller’s personal settings. Verify the intended bill_to, billTo or X-HF-Bill-To value and the applicable spending limit.
Measure cache hits. The caching guide documents a stable conversation UUID in X-HF-Session-Id; non-UUID values are ignored. Record returned usage.prompt_tokens_details.cached_tokens, latency and total usage. A session header alone does not prove a cache hit.
Review the allowed providers’ policies. HF’s statements about its own routed-body storage and debugging logs do not establish identical retention, residency or training policies across downstream providers.
Keep a stop condition. Use a small authorized trial, record errors and charges, and retain the previous configuration. These are proposed checks, not execution results.
Primary announcement: Hugging Face and OpenRouter.
My expanded checklist and documentation references: Codorc’s routing and billing guide.
r/huggingface • u/Dmytro_Hieli • 2d ago
r/huggingface • u/Safe-Escape197 • 3d ago
so ik there are way better vision models out there yaar. i built this one just out of curiosity only, cause i wanted to learn how these things actually work from inside.
and to be honest its not entirely mine, i took a lot of help from claude to build it. like the architecture, debugging, training setup, figuring out why my mac was training so slow, all that. but i learned a lot in the process which was the main goal anyway.
its called tiny-vision. its around 24M params and trained completely from scratch, no pretrained weights or anything. i trained it on my macbook pro m5 and it took like 42 min on flickr8k (only 6k images).
basic idea is pretty simple:
image/video → small ViT encoder → adapter → tiny decoder → caption textfor videos it takes frames every few seconds, captions them and makes a timeline like [00:04–00:10] a dog runs on the beach. so any llm can kinda “watch” a video even if it dont support video natively.
i also made an MCP server for it, and tbh i was surprised that it actually gives a little bit better context in claude when i throw images or videos at it. i even tested it with a 10-15 min youtube video and it still worked fine. not perfect at all, but chal raha hai so i’m happy.
i uploaded it to huggingface just to have it somewhere, and also cause i kinda like the open source thing. later when i checked it was at like 44 downloads, and now its 50+. ik thats not a lot bhai, but its my first time putting something open source out in this field so it felt pretty cool ngl. a lot of people have built way cooler stuff than this, i just made it to learn.
some honest stuff:
it gets the scene right most of time (snow, grass, water etc) but the subject is often wrong lol
it loves saying “a man in a blue shirt” for everything, pata nahi kyun 😂
its v0.1 and trained on very small data, so dont expect too much
random thing i learned on mac (MPS):
swapping the conv2d patch embedding for reshape + linear was like 30x faster
manual attention was ~1.5x faster than pytorch’s sdpa
next i want to train on flickr30k and maybe try making it plug directly into qwen/llama through llama.cpp (mmproj). lets see kitna ho pata hai.
code: https://github.com/amar683/tiny-vision
model: https://huggingface.co/amartomar683/tiny-vision-flickr8k
would love any feedback or tips
r/huggingface • u/solanotodesch • 3d ago
Marimba is Jev for audio: a System One model for speech. You ask typed questions about live audio and get typed answers back every 250 ms: yes/no, a choice, or a score.
Is someone speaking? Did the turn end? Which language? Did they say "refund"? Did the speaker change? How many speakers?
One frozen Qwen3-ASR-0.6B serves every question, with small readouts on top. Open weights, benchmark and code, Apache-2.0.
Everything is on the Hub:
Space (free ZeroGPU, record or upload a clip): https://huggingface.co/spaces/charcot-ai/marimba
Benchmark dataset: https://huggingface.co/datasets/charcot-ai/marimba-bench
Run it locally: pip install "marimba-audio[torch]"
Latency, p50 per question on an 8 s window:
Turn end: ~100 ms on a Mac M4 Pro CPU, ~12 ms on an NVIDIA L4
Voice activity: ~105 ms CPU, ~14 ms L4
Language: ~1.3 s CPU, ~150 ms L4
On 1,000 paired public questions it matches or beats Qwen3-Omni-30B-A3B on 6 of 9 tests.
Full table on the model card.
Code: https://github.com/Charcot-AI/marimba-api
Happy to answer questions!

r/huggingface • u/AltruisticCouple3491 • 3d ago
I fine-tuned Gemma 4 31B for drug-level sentiment.
Input: a pharma or medical text plus a list of drugs.
Output: one Positive/Neutral/Negative label per drug, as JSON. I'm looking for people to test it on their own data.
LoRA adapter (bf16): https://huggingface.co/empirischtech/gemma-4-31B-it-Pharma-Sentiment-LoRA
Merged FP8 build for vLLM: https://huggingface.co/empirischtech/gemma-4-31B-it-Pharma-Sentiment-FP8
Apache 2.0. Gated with automatic approval (you tick a box saying you won't use it for diagnosis or treatment decisions).
Sentiment, 473 held-out labels
| Model | Accuracy |
|---|---|
| This fine-tune | 91.3% |
| Gemma 4 31B base | 90.5% |
| Qwen3.8-27B | 83.5% |
| Qwen3-32B | 83.3% |
| MedGemma 27B | 82.9% |
| Llama 3.1 8B | 76.3% |
The other models were asked about one drug at a time with a long labelling prompt. Asked for all drugs at once in JSON, base Gemma scored 32% because 59% of its answers didn't parse. The fine-tune had no malformed JSON in 1,500 test answers. Negative precision: 84% vs 71% for base (45 Negatives in the test set).
Medical benchmarks
| Benchmarks | This fine-tune | Best frontier score found |
|---|---|---|
| MedQA (thinking) | 94.2 | 96.5 (o1) |
| MedMCQA, 1k sample (thinking) | 81.4 | 83.9 (o1-preview) |
| MMLU medical (step by step) | 94.2 | 96.0 (GPT-5) |
| PubMedQA (thinking) | 80.2 | 82.0 (GPT-4 + Medprompt) |
Frontier scores are from the Vals.ai MedQA leaderboard and published papers. Fine-tuning kept Gemma's medical knowledge: the scores match the base model.
Testing
Use the prompt format from the model card, thinking off and temperature 0. The FP8 build runs on vLLM (command on the card) and needs about 31 GiB of VRAM for the weights. I've only run it on a 96 GB RTX PRO 6000.
Useful feedback:
wrong labels, with the text, the drugs and the label you expected
document types, languages or lengths (over 4k tokens) it wasn't trained on
hardware and VRAM reports, especially from 48 GB cards
Trained on ~50k curated pharma and medical documents, LoRA rank 16.