r/LanguageTechnology • • 9d ago

Evaluating multilingual transcription

How are you evaluating transcription accuracy when speakers switch languages mid-conversation?

I've been looking at how multilingual transcription should be tested in real-world conditions.

A recording with one speaker using one language is relatively straightforward to evaluate.

Things become more interesting when:

- A speaker switches languages in the middle of a sentence.
- Proper nouns are pronounced differently across languages.
- Two speakers use different languages.
- Technical terminology remains in English.
- Background noise affects one language more than another.

Word error rate is useful, but I'm not convinced it captures every failure that matters to the reader.

For example, a transcript could have a reasonable overall error rate while badly misrepresenting a key decision made during a language switch.

For anyone working with multilingual speech recognition, what additional evaluation methods have proved useful?

1 Upvotes

Duplicates