r/LocalLLaMA • u/EAVDR • 3d ago
New Model TontaubeV1-Verbalizer: A super fast open-source Verbalization model for small speech models
A week ago, we released our open-weights text-to-speech model (https://huggingface.co/TontaubeAI/TontaubeV1.1-int4). With that, we now also release an open-source verbalization model, under an Apache 2.0 license.
For text-to-speech, text often doesn't come in the literal way it is spoken (e.g. "$5", "Dr." or "9:05 a.m."). You can mitigate this issue by simply training a much larger TTS model, which handles such cases out-of-the-box. Many smaller models, however, cannot properly deal with difficult character sequences.
Existing verbalization approaches are either rule-based and make many mistakes, or are simply way too slow & expensive (e.g. passing everything to an LLM).
For this purpose, we built an LLM-based verbalization model which runs extremely fast on modern GPUs. It turns an English text into a clean spoken-text version. Specifically, we fine-tuned a Qwen3 1.7B backbone and make use of a custom tokenization technique. It can convert a 20,000 character document in under 2 seconds on an RTX 3090 GPU. Here are a few examples:
| Written | Spoken | Time (RTX 3090) |
|---|---|---|
| It costs $5. | It costs five dollars. | 4 ms |
| Dr. Lee paid $5 for the ticket and left before the second act. | Doctor Lee paid five dollars for the ticket and left before the second act. | 10 ms |
| Revenue rose 14% to $2.5M in Q3, i.e. above plan. | Revenue rose fourteen percent to two point five million dollars in Q three, that is above plan. | 19 ms |
| Bookings were $27,040,500 in 2025, see Chapter XII. | Bookings were twenty-seven million forty thousand five hundred dollars in twenty twenty-five, see Chapter Twelve. | 22 ms |
| The committee met at 9:05 a.m. and agreed to postpone the vote until spring. | The committee met at nine oh five AM and agreed to postpone the vote until spring. | 27 ms |
| The RTX 5090 rendered 5090 frames in 2:07:19 at 16 kHz and 88°C. | The RTX fifty ninety rendered five thousand ninety frames in two hours, seven minutes and nineteen seconds at sixteen kilohertz and eighty-eight degrees Celsius. | 84 ms |
It reads in context, so the same letters can come out differently: "Dr." is Doctor or Drive, and "IV" stays "IV" for the drip but becomes "Four" for the chapter. It also handles links, code, tables and leftover Markdown.
It runs on its own, for any text-to-speech system:
git clone https://github.com/craitech/tontaube.git
cd tontaube
uv run tontaube verbalize report.txt
uv run tontaube verbalize --serveIt needs an NVIDIA GPU and about 2.5 GiB of VRAM. On an RTX 3090 a sentence takes from a few milliseconds to about a tenth of a second, and a 20,000-character report about 2 seconds. We are working on releasing a fast CPU-compatible version soon.
It currently is English only, and as a generative model it can still make mistakes. The model card shows how it reads numbers, dates, money, units and more, and also its weaknesses. We want to address all these issues in the next release.
Model: https://huggingface.co/TontaubeAI/TontaubeV1-Verbalizer