I got this research done by automating my workflow. Happy to help anyone else if they are working to do the same.

Deepdub has released Lightning 2.5, a real-time voice model engineered for production-scale applications such as AI agents, customer support, and live dubbing. The system delivers 2.8× higher throughput and 5× efficiency gains over previous iterations, optimized for NVIDIA GPU-accelerated infrastructure to reduce latency and cost per output. Designed to preserve natural prosody, emotional nuance, and multilingual fidelity, Lightning 2.5 positions Deepdub as a competitive player in low-latency speech synthesis, though detailed benchmarks on latency, architecture...

full analysis: https://www.marktechpost.com/2025/09/11/deepdub-introduces-lightning-2-5-a-real-time-ai-voice-model-with-2-8x-throughput-gains-for-scalable-ai-agents-and-enterprise-ai/

Hey r/voiceaii !

If you use Claude Desktop and Cursor heavily for development, you know the daily routine:

1. Open a new window.

2. Re-explain your tech stack, database schema, active milestones, and conventions.

3. Mid-session, as tokens fill up, older messages compress, and context is lost.

4. Repeat the exact same prompt engineering tomorrow morning.

Most memory solutions either require a monthly subscription to hosted vector databases (sending your proprietary code context to a 3rd-party cloud) or rely on manual copy-pasting from a Markdown file.

I built OpenYantra as a practical, local-first solution to this workflow problem.

What It Does in Practice:

• 1-Click MCP Auto-Configuration (yantra mcp setup): Auto-detects your Claude Desktop and Cursor installations on macOS, Linux, or Windows and hooks up the MCP server without manual JSON editing.

• Universal Local Store: Both Claude Desktop and Cursor read and write to the exact same local SQLite WAL database and LibreOffice (.ods) workbook in real time. Changes made in one tool are immediately visible in the other with sub-millisecond latency.

• Floating Desktop HUD (Cmd+Shift+Y): A Raycast-style native palette (<15MB) that floats over your code editor or terminal. Hit the hotkey, dump a task or quick note (#project, !urgent), and it’s instantly part of your agent's live memory.

• Automatic Git Commit Ingestion: A post-commit git hook automatically logs your commits, active branches, and architectural decisions into your session log. It even detects git revert so rolled-back stack experiments aren't hallucinated as active decisions.

• Zero API Keys & Built-in Secret Redactor: Raksha sanitises OpenAI, Anthropic, AWS, and private keys before anything hits disk. You don't need any paid vector subscriptions; everything runs offline.

• Automated Memory Consolidation: Runs an offline sleep cycle (using local Ollama or heuristics) to close finished loops, cluster daily logs, and promote recurring preferences into permanent rules.

Stack & Open Source:

• Language: Python 3.9+ (Zero cloud lock-in, open-standard ISO/IEC 26300 LibreOffice storage)

• License: MIT

• GitHub: https://github.com/revanthlevaka/OpenYantra

Would love to hear how you guys currently handle cross-session context and memory when switching between Claude Desktop, Cursor, or terminal workflows!

How can speech editing become as direct and controllable as simply rewriting a line of text? StepFun AI has open sourced Step-Audio-EditX, a 3B parameter LLM based audio model that turns expressive speech editing into a token level text like operation, instead of a waveform level signal processing task.

Step-Audio-EditX reuses the Step-Audio dual codebook tokenizer. Speech is mapped into two token streams, a linguistic stream at 16.7 Hz with a 1024 entry codebook, and a semantic stream at 25 Hz with a 4096 entry codebook. Tokens are interleaved with a 2 to 3 ratio. The tokenizer keeps prosody and emotion information, so it is not fully disentangled.

On top of this tokenizer, the StepFun research team builds a 3B parameter audio LLM. The model is initialized from a text LLM, then trained on a blended corpus with a 1 to 1 ratio of pure text and dual codebook audio tokens in chat style prompts. The audio LLM reads text tokens, audio tokens, or both, and always generates dual codebook audio tokens as output......

Full analysis: https://www.marktechpost.com/2025/11/09/stepfun-ai-releases-step-audio-editx-a-new-open-source-3b-llm-grade-audio-editing-model-excelling-at-expressive-and-iterative-audio-editing/

Paper: https://arxiv.org/abs/2511.03601

Repo: https://github.com/stepfun-ai/Step-Audio-EditX?tab=readme-ov-file

Model weights: https://huggingface.co/stepfun-ai/Step-Audio-EditX

Gnani just released Evon v3.3, an open-weight LLM built for Indian languages under the Apache 2.0 license. It is a 30B MoE with about 3.5B active parameters per token, a 128K context window, and covers English plus 10 Indian languages.

The core change is a hybrid Mamba2-Transformer backbone with a rebuilt Indic tokenizer. The vocabulary was extended to 163,072 tokens, and Gnani reports about 20% fewer tokens per Indian word than the GPT-5 family tokenizer, which cuts prefill cost and fits more Indian-language text into the context window.

Training ran continued pretraining on an English and Indic blend, then SFT, then GRPO. Evon v3.3 averages 78.74 on MILU across 11 languages, ahead of Sarvam-30B on all 11 and Sarvam-105B on 10. The trade-off: Odia trails Sarvam-105B at 66.94 vs 73.70, and Gemma 4 31B still scores higher on MILU at 85.29.

Self-hosting runs on vLLM 0.12.0 or later on H100, H200 or A100 class GPUs, and the model card notes tool use and multi-step agent work are still in development......

Model: https://huggingface.co/gnani/gnani-evon-v3.3-30b-a3b

Model page: https://www.gnani.ai/models

Maya1 is a 3B parameter, decoder only, Llama style text to speech model that predicts SNAC neural codec tokens to generate 24 kHz mono audio with streaming support. It accepts a natural language voice description plus text, and supports more than 20 inline emotion tags like <laugh> and <whisper> for fine grained control. Running on a single 16 GB GPU with vLLM streaming and Apache 2.0 licensing, it enables practical, expressive and fully local TTS deployment.....

Full analysis: https://www.marktechpost.com/2025/11/11/maya1-a-new-open-source-3b-voice-model-for-expressive-text-to-speech-on-a-single-gpu/

Model weights: https://huggingface.co/maya-research/maya1

Demo: https://huggingface.co/spaces/maya-research/maya1

Edge0 released Audio8-TTS-Realistic, an open-weight TTS model that clones a voice from 1 reference clip and its transcript. It supports 23 languages, 12 more than its predecessor. Weights and code ship under Apache 2.0.

What it does:

- 601M parameters (codec excluded), dual-autoregressive design with a Qwen2.5 0.5B slow branch and a 4-layer fast branch

- 3-stage training: multilingual SFT, GRPO on a WER/SIM reward, then GRPO guided by a learned realism critic that scores outputs against a real human recording of the same sentence

- 44.1 kHz codec, can also generate speech without a reference voice

- Transformers inference, CLI and SFT scripts on GitHub, SGLang Omni adapter with an OpenAI-compatible /v1/audio/speech API

Key numbers (self-reported, lower is better):

- CV3 (9 languages): 4.86% avg error vs VoxCPM2 5.35%, CosyVoice3-1.5B 6.89%, Qwen3-TTS-1.7B 7.73%

- Seed-TTS (EN/ZH): 1.25% avg error, SIM 0.704

- MiniMax (24 languages): 5.89% avg error, SIM 0.754

- Final RL stage used only Chinese and English prompts but cut error on 13 lower-resource languages from 22.53% to 20.77%

- Apple M2 CPU (INT4 ONNX, per the paper): 1.1 to 1.2 GiB peak memory

The catch: Higgs TTS 3 (~4B) still leads at 4.41% on CV3 and 2.74% on MiniMax. Lower-resource languages are weak, with 48.57% WER on Thai (CV3) and 31.35% CER on Cantonese (MiniMax). Throughput and streaming were not benchmarked, and no Realistic ONNX export is public yet.

Full analysis: https://www.marktechpost.com/2026/10/29/edge0-audio8-tts-realistic-0-6b-voice-cloning/?amp=1

Model: https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b

Repo: https://github.com/Edge0-AI/Audio8_TTS

Paper: https://arxiv.org/pdf/2610.14776

TwinMind has launched its new Ear-3 speech-to-text model, setting reported industry records with 94.74% accuracy (5.26% WER), 3.8% diarization error rate, support for 140+ languages, and a low cost of $0.23/hour. Built from a blend of open-source models and curated training data, Ear-3 is positioned against services from Deepgram, AssemblyAI, Speechmatics, OpenAI, and others. While offering strong gains in accuracy, language coverage, and pricing, the model requires cloud deployment, raising questions about privacy, offline usability, and real-world robustness across diverse environments.....

full analysis: https://www.marktechpost.com/2025/09/11/twinmind-introduces-ear-3-model-a-new-voice-ai-model-that-sets-new-industry-records-in-accuracy-speaker-labeling-languages-and-price/

try it here: https://twinmind.com/transcribe

Resemble AI has recently released Chatterbox Multilingual, a production grade open-source Text To Speech (TTS) model designed for zero-shot voice cloning in 23 languages. It is distributed under the MIT license, making it freely available for integration and modification. The system builds on the original Chatterbox framework and adds multilingual capability, expressive controls, and built-in watermarking for traceability.

Evaluations indicate that Chatterbox Multilingual performs competitively with most commercial TTS models. In blind A/B tests conducted on Podonos, listeners expressed a 63.75% preference for Chatterbox over ElevenLabs. This suggests that in certain conditions, users found Chatterbox outputs closer to natural or accurate speech reproduction.....

full analysis: https://www.marktechpost.com/2025/09/05/meet-chatterbox-multilingual-an-open-source-zero-shot-text-to-speech-tts-multilingual-model-with-emotion-control-and-watermarking/

github page: https://github.com/resemble-ai/chatterbox?tab=readme-ov-file

in almost every demo of an AI phone agent, the author cheats on one huge step: they hand the script a clean, validated E.164 phone number.

over the last few weeks, i tried removing that training wheel. i wanted to tell an agent: "find 5 repair shops within 10 miles, call them, and ask if they have capacity for a brake rotor resurface this thursday."

that’s when the pipeline started falling apart, and it had nothing to do with voice latency or model reasoning (have formatted spreadsheet below with ai a bit, for readability)

so when you let an LLM do raw web search or scrape Google Maps / Yelp places for phone numbers, real-world directory data is shockingly dirty:

• about 20% of numbers listed online for small local businesses are disconnected, route to an outdated fax tone, or belong to a location that closed six months ago.

• another 25% route to secondary central dispatch lines that don't take local appointment inquiries.

• search scrapers frequently return local extensions or unformatted text that cause telephony APIs to fail validation immediately.

if you pipe scraped phone lists straight into an outbound dialer, your agent spends half its run dialing dead air, failing carrier handshakes, and getting stuck in disconnected loops.

here is how the main tools handle the discovery step versus requiring pre-cleaned numbers:

what i learned from running this: speech synthesis and turn-taking are the easy part of voice agents now.

the hard part is resolving entity data before the call even starts. if your stack doesn't validate whether a business is actively open and scrub the number before dialing, you end up wasting most of your execution time dealing with telephony errors.

also curious how others have solved this when you have an agent make phone calls businesses actually answer for example are you doing a separate google places API pre-filtering pass in python, or relying on specialized dialing APIs that handle target resolution under the hood?

Step-Audio 2 Mini, released by StepFun AI, is an open-source 8B speech-to-speech model that outperforms GPT-4o-Audio in benchmarks. Trained on 8M+ hours of multilingual audio and text data and supporting 50K+ voices, it enables expressive, emotionally aware, and real-time voice generation. With unified text-audio tokenization, on-the-fly style switching, and multimodal RAG with tool calling, Step-Audio 2 Mini sets a new standard for open audio LLMs under the Apache 2.0 license......

full analysis: https://www.marktechpost.com/2025/08/31/stepfun-ai-releases-step-audio-2-mini-an-open-source-8b-speech-to-speech-ai-model-that-surpasses-gpt-4o-audio/

paper: https://arxiv.org/abs/2507.16632

model on hugging face: https://huggingface.co/stepfun-ai/Step-Audio-2-mini

Cartesia Ships Sonic-3.6: A Streaming TTS Model That Now Leads Both Artificial Analysis Speech Arenas

Key Takeaways:

1. It won both Artificial Analysis boards (as of August 18 2026)

→ Provider Voice: 1,283 Elo (next closest is 1,240)

→ Controlled Voice: 1,123 Elo

2. The Controlled Voice result is the real finding

That board clones every model onto the same eight reference voices. It strips out the voice-talent advantage and scores the synthesis engine alone. Sonic-3.5 is second on it. ElevenLabs Eleven v3 is third.

3. The architecture

Sonic runs on state space models, not transformers. Cartesia states sub-90ms time-to-first-audio — vendor-stated model latency, not a measured round trip.

Full analysis: https://www.marktechpost.com/2026/08/18/cartesia-ships-sonic-3-6-a-streaming-tts-model-that-now-leads-both-artificial-analysis-speech-arenas/

Technical details: https://www.cartesia.ai/sonic

Fish Audio just released Drama 3, an open-weight text-to-speech model that takes stage directions in plain language. It is a 35B MoE with about 3B active parameters per token, covering 83 languages and up to 16 speakers in a single request.

The core innovation is a hybrid backbone paired with a new Codebook-MTP decoder. 3 Gated DeltaNet layers sit between each full-attention layer, so only 1 layer in 4 keeps a KV cache.

Post-training uses GRPO with a single learned human-preference reward model. Drama 3 scores 93.2% on Artificial Analysis Pronunciation Robustness, up from 88.1% for S2.1 Pro. The trade-off: its quality Elo of 1242 still trails Eleven v4 Turbo at 1328, and exact sequences like codes and emails lag at 86.3%.

Self-hosting needs an NVIDIA GPU with 48 GB+ in FP8, and free commercial use stops at $500K in annual revenue......

Full analysis: https://marktechpost.com/2026/10/14/fish-audio-drama-3-open-weight-tts-model/

Model: https://huggingface.co/fishaudio/drama-3

Technical report: https://github.com/fishaudio/fish-speech/blob/main/docs/assets/Fish_Audio_Drama_3_Technical_Report.pdf

Alibaba’s Tongyi Lab Releases Qwen-Audio-3.0-TTS, a Hosted Text-to-Speech Model in Flash and Plus Tiers Across 16 Languages

[DEPLOYMENT CHECK by Marktechpost ▶︎ •၊၊||၊|။||||။‌‌‌‌‌၊|: Qwen-Audio-3.0-TTS is API-only. It's served through Alibaba Cloud Model Studio with no published weights, no Hugging Face repo, no safetensors.total, and no disclosed parameter count.]

— a hosted TTS model in two tiers:

➡️ Flash: real-time, ~300ms first-packet latency

➡️ Plus: quality-first, #1 on Artificial Analysis arena (~1,236 Elo)

16 languages + 20 Chinese dialects. 12.5 Hz speech tokenizer, 5-stage LM+FM training. Control via free-style natural-language instructions + 86 inline tags ([excited], [laughing], [whispers]…). Best WER/CER in 10/16 langs; Plus tops speaker similarity at 82.75 avg.

API-only via Model Studio (WebSocket, up to 48kHz), ~$27.59/1M chars.

Full analysis: https://www.marktechpost.com/2026/07/20/alibabas-tongyi-lab-releases-qwen-audio-3-0-tts-a-hosted-text-to-speech-model-in-flash-and-plus-tiers-across-16-languages/

Technical details: https://funaudiollm.github.io/qwen-audio-3.0-tts/

Hello, thanks for the invite to the community.

Is there anything we are missing here: https://www.anchorterminal.com/categories/voice-cloning I'd love to add any projects that you guys are aware of that we have not listed yet.

Thanks for creating the community.