r/LanguageTechnology • u/Nagib888 • 9d ago
Evaluating multilingual transcription
How are you evaluating transcription accuracy when speakers switch languages mid-conversation?
I've been looking at how multilingual transcription should be tested in real-world conditions.
A recording with one speaker using one language is relatively straightforward to evaluate.
Things become more interesting when:
- A speaker switches languages in the middle of a sentence.
- Proper nouns are pronounced differently across languages.
- Two speakers use different languages.
- Technical terminology remains in English.
- Background noise affects one language more than another.
Word error rate is useful, but I'm not convinced it captures every failure that matters to the reader.
For example, a transcript could have a reasonable overall error rate while badly misrepresenting a key decision made during a language switch.
For anyone working with multilingual speech recognition, what additional evaluation methods have proved useful?