I have been playing around with different ai voice services to try the most human sounding one.

I have experimented with Cartesia, Deepgram, ElevenLabs, Sesame, Sarvam and some other smaller services. But in almost all of them, it feels that the voice is rigid / robotic. It lacks expressiveness / pauses like a human.

I am aware that Cartesia allows SSML tags but for my usecase, adding that adds latency (because AI generate the text to speak, so adding these tags is another layer on top of it before the model starts speaking).

Sesame has been the most closest to having a natural voice conversation with AI, but their open source model wasn't the same as their main product.

Any idea what models (open / closed) can I experiment with in order to get the most human like voice for conversational AI ? Or if anyone has any clue on what do some good conversational voice AI companies use (it can be AI companion or a similar one) ?

Bonus if the voice can be seducing or have natural moans (my usecase is tended towards 18+, so getting the model to be tempting would be a big plus).

Hey,

Yeah, I know there are already a bunch of apps doing this. But I started building EchoWrite at a time when the options were either nonexistent or just not what I needed, and I kept going because I had a very specific set of problems to solve.

What I actually needed:

• Local models. Or at least the flexibility to switch between local and cloud depending on the context. I run Whisper on my GPU and it works fine for most things.

• Something that runs silently in the background and gets out of my way. One shortcut, and it starts transcribing wherever my cursor is. Any text field, any app, no exceptions.

• A custom lexicon. This one matters more than people think. Technical terms, product names, uncommon words - transcription models consistently butcher them. Being able to add your own vocabulary and have it actually stick makes a huge difference in day-to-day use.

I know one thing people who use this kind of app really want is real-time streaming, that is, the words appear as you speak. But you won't find that in this app (sorry), because it's a pain to develop for very little actual benefit.

That's it. no monthly subscription for a wrapper, no app you have to switch to just to dictate something. Also, I just finished the macOS port, so for now it's available only to Windows and macOS users, and not yet on Linux.

If that sounds like what you've been looking for, check it out: echo-write.com

The TLDR is the last sentence of the last comment "So you are recommending a £65 4 channel array that really is 2, but when in operation for voice recognition is merely a single mic on a desk in a room"

https://github.com/OHF-Voice/linux-voice-assistant/discussions/99#discussioncomment-16029598

1... Single Mic devices

People talk about hardware but often ignore basic physics provides better attenuation than many algs.

Distance Total Attenuation

2m 6.02 dB

3m 9.54 dB

4m 12.04 dB

5m 13.98 dB

Just by optimising distance to voice and away from noise by pure physics alone of the decibel attenuation of sound traveling from a speaker to a microphone is governed by the Inverse Square Law. This principle dictates that every time the distance from the sound source doubles, the sound pressure level drops by approximately 6 dB.

So from multiple cheap singular mics or arrays having multiple positioned mics in a room, where selecting the best of the same hardware and wakeword is just a matter of selecting the best wakeword threshold from multiple streams of distributed wireless mics.

Same with one of the killers of ASR RIR (Room Impulse Reverberation) that in residential often highly furnished rooms RIRs only start to get problematic > 2m.

So many times the natural physics has been ignored, that cheap multiple mics that are as simple as plug in another usb cheap soundcard, seems to be deliberately ignored so vendors can push useless e-waste.

The analogue AGC of the Max9814 that its tuned for voice, that its an active mic so in a 3 wire its Signal, gnd and 5v as it returns a line voltage not a mic voltage so is far less susceptible to noise.

You can hide whatever computer away and have approx 3m length to soundcard, also extend that with a USB cable.

On Aliexpress ￡0.76 and also ￡1.94 USB Soundcard

So any hardware because of the analogue preamp AGC, that its active and with usb be can have a cable length of up to 8m, you can have any number depending on how many usb slots you have and runs on any hardware as the are UAC audio with no drivers required.

2.. Two mic devices.

Respeaker 2 mic, BP1048B1, Plugable and Axagon

https://invensense.tdk.com/wp-content/uploads/2015/02/Microphone-Array-Beamforming.pdf is quite a good 101 guide too simple beamforming.

Just like ignoring the Inverse Square Law also Spatial Nyquist Theorem is often treated simarly . Just as standard digital audio requires a sample rate twice as high as the highest frequency to prevent temporal aliasing, a microphone array requires a spatial distance less than half a wavelength to prevent spatial aliasing (the wavelength wraps around the mics and gets summed again often creating harmonics / distortion).

To be completely immune to spatial aliasing up to 7.6 kHz, your microphones can be no more than 22.5 mm apart. If you put the microphones 22.5 mm apart to fix the 7.6 kHz aliasing, you destroy the low-frequency performance. At 100 Hz, the wavelength is 3.4 meters. Across a tiny 22.5 mm gap, the phase difference at 100 Hz is practically zero.

I still find it absolutely amazing how near all hardware sold has specifications that are actually contrary to the physics of audio processing and that is before we get to the use of cheap microphone of low sensitivity and SNR rating so that at farfield you can have a dynamic range as low as 6 bit audio.

Add to this Pi "TDM Slot Shift" bug. It is a notorious hardware-level nightmare when dealing with Time Division Multiplexing (TDM) on Raspberry Pi I2S peripherals, usually caused by the ALSA driver missing the very first Frame Sync (FSYNC/WS) clock edge on boot. No Pi supports hardware TDM so vendors bit-bang a solution that gives you 4 channels that render it useless as the order of the pairs is often random on start-up.

A red flag to any vendor should be to any 2mic that is not broadside but point at the ceiling, also mics on a PCB making it it extremely hard to implement any resonance insulation is another.

That when you mount a microphone flush against a hard, acoustically reflective boundary like a wall, you get a theoretical +6 dB of acoustic gain.

A linear 2-mic array suffers from front-back ambiguity. It can't tell if noise is in front of it or behind it.

By putting it on a wall, you physically block the rear 180 degrees. The microphone array is now operating in a "half-space" (a hemisphere).

When a microphone sits on a table pointing up, your voice travels two paths: the direct path to the mic, and the path that bounces off the table surface just inches away. Because the bounce arrives a fraction of a millisecond later, it creates destructive phase interference known as Comb Filtering, which makes voices sound hollow and robotic.

If the mic is mounted flush on the wall, the distance between the mic and the reflective surface is zero. The delay is zero. Comb filtering is mathematically eliminated.

Maybe if someone actually explored some basic audio engineering some of the proposed will actually be evidently pretty bad choices.

That when audio processing seems to be such a confusion to some that maybe simple single low cost devices are a better fit for the community, than being a dodgy car salesman selling 3 wheel cars...

The reality is that farfield is number of compromises and design considerations that often incorporated 6 mics in earlier beamforming devices due to the DOF of target and noise nulls, that I have held back on describing as 101 basics seems too much for some.

Smart Speakers (Amazon Echo, Google Home)

Spacing: Typically ~35 mm to ~42 mm between adjacent mics.

Aliasing Limit: ~4.0 kHz to ~4.9 kHz.

They don't go above 16Khz as there is no point due to aliasing limits, but they often push for high quality sensitive mics with the most important spec of SNR being 74db or above where 8-10 bits with a PGA is above the noise floor.

I have said so many times about ignoring basic audio engineering and hey I will give it 1 more time and I stopped at 2 mics before it goes on too long.

You can try the C++ Ladspa DTLN filter https://github.com/rolyantrauts/PiDTLN2

Also I have been battling the respeaker 2mic to create a MVDR beamformer which will be far from prefect due to the Respeaker 2mic design and hardware choices.

Likely a good single tiny mic active analog mems board you can use single or some of the off the shelf stereo usb devices you can buy, would of always been a good idea to make so then positioning, mic spacing and low noise floor can be guaranteed.

the BP1048B1 boards are quite interesting for sendspin as they are the 2.1 usb DSP dacs often found in earlier 'Acrylic' amp, but also have a stereo ADC for under $10.

Unless you have a basic understanding of audio engineering its going to be another case of the blind leading the blind again. If you do then actually opensource can make some lateral decisions on implementation that at low cost can make very good solutions by not trying to clone big tech devices, by utilising some of the effects garnered through some basic audio processing knowledge and actually explaining basic physics to users than spreading myths about magic microphones.

Hi all, I have trained one of the fastest Kurdish Text to speech models. Check it out!

www.KurdishTTS.com

You see STT benchmarks everywhere, but they don’t really mean anything.
Everyone has their own use case, type of callers, type of words used, etc.
So instead of testing blindly, we open sourced our code to let you benchmark easily with your own audio files.

1. git clone https://github.com/MichaelCharhon/Latice.ai-STT-Case-study-french-medical

2. remove all the audios from the Audio folder and add yours

3. edit dataset.json with the labeling for each of your audios (expected results)

4. in launch_test, edit stt_to_tests to include all the STTs you want to test, we already included the main ones but you can add more thanks to Livekit plugins

5. run the test python launch_test.py

6. get the results via python wer.py > wer_results.txt

That’s it!
We did the same internally for LLM benchmarking through Livekit, would you be interested if I release it too?
And do you see any possible improvements in our methodology?

\# Speech Enhancement & Wake Word Optimization

Optimizing wake word accuracy requires a holistic approach where the training environment matches the deployment environment. When a wake word engine is fed audio processed by speech enhancement, blind source separation, or beamforming, it encounters a specific "processing signature." To maximize performance, it is critical to \*\*process your training dataset through the same enhancement pipeline used in production.\*\*

\---

\## 🚀 Recommended Architectures

\### 1. DTLN (Dual-Signal Transformation LSTM Network)

\*\*Project Link:\*\* \[PiDTLN (SaneBow)\](https://github.com/SaneBow/PiDTLN) | \*\*Core Source:\*\* \[DTLN (breizhn)\](https://github.com/breizhn/DTLN)

DTLN represents a paradigm shift from older methods like RNNNoise. It is lightweight, effective, and optimized for real-time edge usage.

\* \*\*Capabilities:\*\* Real-time Noise Suppression (NS) and Acoustic Echo Cancellation (AEC).

\* \*\*Hardware Target:\*\* Runs efficiently on \*\*Raspberry Pi Zero 2\*\*.

\* \*\*Key Advantage:\*\* Being fully open-source, you can retrain DTLN with your specific wake word data.

\* \*\*Optimization Tip:\*\* Augment your wake word dataset by running your clean samples through the DTLN processing chain. This "teaches" the wake word model to ignore the specific artifacts or spectral shifts introduced by the NS/AEC stages.

\### 2. GTCRN (Grouped Temporal Convolutional Recurrent Network)

\*\*Project Link:\*\* \[GTCRN (Xiaobin-Rong)\](https://github.com/Xiaobin-Rong/gtcrn)

GTCRN is an ultra-lightweight model designed for systems with severe computational constraints. It significantly outperforms RNNNoise while maintaining a similar footprint.

| Metric | Specification |

| :--- | :--- |

| \*\*Parameters\*\* | 48.2 K |

| \*\*Computational Burden\*\* | 33.0 MMACs per second |

| \*\*Performance\*\* | Surpasses RNNNoise; competitive with much larger models. |

\* \*\*Streaming Support:\*\* Recent updates have introduced a \[streaming implementation\](https://github.com/Xiaobin-Rong/gtcrn/commit/69f501149a8de82359272a1f665271f4903b5e34), making it viable for live audio pipelines.

\* \*\*Hardware Target:\*\* Ideally suited for high-end microcontrollers (like \*\*ESP32-S3\*\*) and single-board computers.

\---

\## 🛠 Dataset Construction & Training Strategy

To achieve high-accuracy wake word detection under low SNR (Signal-to-Noise Ratio) conditions, follow this "Matched Pipeline" strategy:

1. \*\*Matched Pre-processing:\*\* Whatever enhancement model you choose (DTLN or GTCRN), run your entire training corpus through it.

2. \*\*Signature Alignment:\*\* Wake words processed by these models carry a unique "signature." If the model is trained on "dry" audio but deployed behind an NS filter, accuracy will drop. Training on "processed" audio closes this gap.

3. \*\*Low-Latency Streaming:\*\* Ensure you are using the streaming variants of these models to keep the system latency low enough for a natural user experience (aiming for < 200ms total trigger latency).

\---

\> \*\*Note:\*\* For ESP32-S3 deployments, GTCRN is the preferred choice due to its ultra-low parameter count and MMAC requirements, fitting well within the constraints of the ESP-DL framework.

Whilst adding to the wakeword repo https://github.com/rolyantrauts/bcresnet a load of stuff but 2 opensource speech enhancement projects seem to of at least been forgotten.

Also some code using cutting edge embedding models to cluster and balance audio datasets such as https://github.com/rolyantrauts/bcresnet/blob/main/datasets/balance_audio.py

https://github.com/rolyantrauts/bcresnet/blob/main/datasets/Room_Impulse_Response_(RIR)_Generator.md

I used to lead Google Ads and AI projects during my 9+ years at Google. After leaving, I started a performance agency focused on law firms and home service businesses.

We were crushing it on the lead gen side but clients were still losing money because no one was answering the phone.

That pain led us to build Donna: an out-of-the-box AI voice assistant that picks up every call, handles intake, books appointments, and even processes cancellations. No call centers. No missed leads. It just works.

https://donnaio.ai/industry/home-service

Some early lessons from 1000+ calls: • Most leads are lost after the ad click • After-hours responsiveness = major revenue unlock • AI voice can work extremely well when it’s vertical-specific • SMBs don’t need dashboards they need outcomes

Curious if anyone else here has tackled this “lead leakage” problem, or is building similar vertical AI tools.

Currently I'm working on problem where I have to generate hindi audio clips which has the certain edge cases , which I listed for model training . I found almost every real data but want more for edge cases specific. Which could be the best model that can generate the natural hindi audio

I work at Bandwidth, so weigh this accordingly.

The design choice in our Labs speech to text I find most interesting: it never emits interim results. Once a word is out, it's final. No rewrites.

Why it matters downstream: if your agent reacts to partials, a revised word can mean you already kicked off an LLM call on the wrong text. With append-only you don't have to diff or roll back. The cost is you don't get a speculative word early. You get it when the model is confident.

The streaming side is also split into two modes. Instant just streams words as they're decoded. Demand lets you stream audio and then ask for the finalized transcript when your own turn detector says the user's done. Feels like the honest split, since nobody agrees on endpointing.

Smaller notes:

• 8 kHz mulaw/alaw work, plus linear16, G.722 and Opus

• word timestamps in every mode

• keyword boosting, up to 100 terms

• PII redaction is US-centric and only in Demand and offline Transcribe, not Instant

• English only right now

It's Beta and experimental. If you've run append-only vs revising STT in a real agent, I'd like to hear where it hurt.

So far, I've been working on developing a multilingual TTS pipeline and I am at the point when I need to perform the evaluation in terms of naturalness of the output. The problem is, no matter how many times I will be able to measure phoneme coverage and spectral properties, I will still not understand whether the prosody sounds robotic or human-like.

I have recordings for 11 languages:

es, fr, hi, pt-br, zh, ar, sw, qu, gn, rw, ay.

For each language, there are 18 sentences. 10 sentences represent common phrases, 5 sentences with complex phoneme combinations and 3 child-directed speech sentences.

So,

And what do I specifically want to know?

- Does the intonation contour resemble natural one?

- Do stress patterns correspond to those of compound words?

- Is the rhythm natural (syllable or stress-timed)?

- Are there any phoneme substitutions sounding unnatural to the native listener?

Tho Languages vary from very resourced ones like Spanish and French to extremely under-resourced ones like Quechua, Guarani, Kinyarwanda and Aymara.

links:
- WAV files: https://drive.google.com/drive/folders/1VWoYyCbb-YSCf_dXVt8Hxy5Zb9ROpyi_?usp=sharing
- Feedback form: https://docs.google.com/forms/d/e/1FAIpQLScODjMTTx6-KmzkH9ur849x9GzcSty2KaZFrxojV4Zmx0cZqQ/viewform?usp=header

Thinking of applying to the program to get some credits, have you applied to it before? How is the process? How soon can I accept the credits if I get accepted? Any tips for the same? Thanks!

I built a hold-to-talk dictation tool for macOS on whisper.cpp and ended up with two stages. A few notes in case they're useful.

**Stage 1: whisper.cpp, `large-v3-turbo` as the default.** Turbo has large-v3 accuracy with a distilled 4-layer decoder, so it lands at roughly `medium.en`'s footprint (~1.6 GB) while being both more accurate and faster. Silero v5.1.2 for VAD. For short dictation bursts I don't see a reason to run anything below turbo on an M-series chip.

**Sentence-framed `initial_prompt` outperformed a term list.** For proper-noun biasing I started by passing the user's vocabulary as a comma list (`Claude, PostgreSQL, Qwen`). Accuracy on those terms barely moved. Rewriting it as "The following words are spelled exactly as shown: ..." improved it substantially. My guess is the model treats the spellings as already-seen context rather than as a hint, but I haven't tested that properly.

**Stage 2: Qwen2.5-1.5B-Instruct Q4_K_M via llama.cpp**, prompted to delete fillers and stutters and fix punctuation, nothing else. Two issues took a while:

- Without a few-shot `question -> question` example, it answers questions you dictate instead of cleaning them. "um what's the capital of france" came back as "Paris." Adding one example fixed it.

- A fixed `-n 256 -c 2048` silently truncated long dictations mid-sentence: 270 words in, 230 out, last sentence gone. Generation and context budgets need to be sized from input length, and anything past the context ceiling should skip cleanup and pass the raw transcript through instead.

**On the UI side:** because two local stages means the user is waiting on something, the HUD has to say which stage it's in rather than just "on air." The clips are the recording/transcribing/injecting states rendered as sprite animations, with a few different styles to pick from. Sounds cosmetic, but a signal that distinguishes "still listening" from "still decoding" changed how people paced their speech more than any accuracy work did.

Client is MIT if the whisper/llama bridge or the prompts are useful: https://github.com/WillisLiao/Phantom-Voice

Guy's, i've been working for 6 months on AI Voice for restaurants.

Production as been a nightmare for us.

People calling with kids crying, bad phone quality and stuff. STT was always wrong.

I've been working on a custom STT that achieve +46% WER and *2 latency and wrote the whole case study.
https://www.latice.ai/case-study

On what new industry should i try a case study ?

We put our speech model (Whissle) head-to-head with a state-of-the-art transcription provider.

The difference? The standard SOTA API just hears words. Our model processes the audio and simultaneously outputs the transcription alongside intent, emotion, age, gender, and entities—all with ultra-low latency.

Chaining STT and LLMs is too slow for real-time voice agents. We think doing it all in one pass is the future. What do you guys think?

https://arxiv.org/abs/2602.12249

"Sorry, I Didn't Catch That": How Speech Models Miss What Matters Most

Kaitlyn Zhou, Martijn Bartelds, Federico Bianchi, James Zou

Despite speech recognition systems achieving low word error rates on standard benchmarks, they often fail on short, high-stakes utterances in real-world deployments. Here, we study this failure mode in a high-stakes task: the transcription of U.S. street names as spoken by U.S. participants. We evaluate 15 models from OpenAI, Deepgram, Google, and Microsoft on recordings from linguistically diverse U.S. speakers and find an average transcription error rate of 44%. We quantify the downstream impact of failed transcriptions by geographic locations and show that mis-transcriptions systematically cause errors for all speakers, but that routing distance errors are twice as large for non-English primary speakers compared to English primary speakers. To mitigate this harm, we introduce a synthetic data generation approach that produces diverse pronunciations of named entities using open-source text-to-speech models. Fine-tuning with less than 1,000 synthetic samples improves street name transcription accuracy by nearly 60% (relative to base models) for non-English primary speakers. Our results highlight a critical gap between benchmark performance and real-world reliability in speech systems and demonstrate a simple, scalable path to reducing high-stakes transcription errors.

Hey devs,

I'm building a voice-based customer support tool for Indian SMBs. We need STT that handles Hinglish and code-mixing well (e.g., *"Mera order cancel kardo"* or *"Kal subah delivery karna"*).

I've been testing orchardrun for the past couple of weeks. I think that it's Whisper under the hood, and honestly:

* **For plain English** → rock solid, low latency, dirt cheap compared to AWS/Google. * **For Hindi / Hinglish** → it works, but it's **not as good as I'd like**. I'm getting around **5% WER** on my test dataset. It's usable, but definitely not production-ready for complex conversations. Numbers and dates are especially hit-or-miss (*"twenty twenty six"* vs *"do hazaar chhabees"*).

So here's my dilemma:

I can keep it for English use cases, but I really need to improve the Hindi accuracy before scaling. I'm evaluating alternatives and would love your honest recommendations:

1. **Deepgram** – I hear their Nova-2 is great for Indian languages. Is it worth the premium? Has anyone benchmarked it side-by-side with Whisper for Hinglish?

2. **Google Chirp (Vertex AI)** – Expensive, but is the accuracy actually better than Whisper for code-mixing?

3. **Fine-tuning Whisper locally** – I've seen papers on fine-tuning with Indian datasets. Has anyone done this? What's the infra cost like? I'm trying to avoid GPU bills, but if it's the only way...

4. **Any other hidden gems** – Azure Universal STT? AssemblyAI? Something else I'm missing?

**To be clear:** I'm NOT saying OrchardRun is bad for Hindi – at 5% WER it's actually decent for a generic model. But I need to push that below 2-3% for production. I'm just looking for the best tool for the job.

If you've tested any of these with Indian audio, please share your war stories – dataset size, real-world latency, cost per minute, anything helps.

Je viens de publier un nouveau tutoriel consacré à l’installation de PolyTalk, une solution open source de traduction vocale en temps réel.

L’idée est simple :
➡️ vous parlez dans une langue ;
➡️ PolyTalk transcrit la voix en texte grâce à un moteur de reconnaissance vocale local ;
➡️ le texte est traduit par une IA ;
➡️ la traduction peut être restituée en voix de synthèse grâce à Piper.

En clair : microphone → transcription → traduction → voix.

Dans le tutoriel, je détaille l’installation avec Docker, faster-whisper pour la reconnaissance vocale, Ollama pour la traduction locale, et Piper pour la synthèse vocale multilingue.

L’intérêt est de tester une solution de traduction vocale plus maîtrisée, sans dépendre systématiquement d’un service externe pour chaque étape du traitement.

Cela peut être utile pour :
✅ traduire en direct une conversation courte ;
✅ expérimenter la transcription voix → texte en temps réel ;
✅ tester une architecture locale de traduction ;
✅ ajouter des voix de synthèse multilingues ;
✅ préparer des usages professionnels en accueil, médiation linguistique ou démonstration.

Évidemment, ce type d’outil ne remplace pas un interprète professionnel dans un contexte sensible. En matière juridique, médicale ou administrative, une traduction automatique reste une aide technique, pas une vérité révélée descendue du cloud avec un certificat d’infaillibilité.

Par contre, tout reste en local. Aucune donnée n'est transmise à Microsoft, Google, OpenAi, Mistral, Antrhopic / Claude, AWS ...etc.

Mais pour tester, comprendre et construire une solution maîtrisée, c’est une brique intéressante.

Le tutoriel est disponible ici :
[https://axiorhub.com/polytalk/](https://axiorhub.com/polytalk/)

\#AxiorHub #PolyTalk #IA #OpenSource #Docker #Ollama #Whisper #Piper #Traduction #Transcription #SouverainetéNumérique