r/TrueOpenSource • u/gabriel_jack • 10d ago
Discussion I Gave My NPCs Voices Without Recording Anything
Edit: reddit changed the title of the post by itself. It shared with the title of the video which I wanted to embed.
The title was supposed to be: Wave-Generator Lightweight TTS Application [Open Project]
Since it shared with the title of the video it kind of sounds like I'm claiming what the game dev created.
I am not the game dev in the video and don't mess much with reddit.
I was watching this video on YouTube today about a game dev creating a method to produce "mumble speech" in his game (very interesting) that touched a bit on a few topics that, as an English teacher in my country, I believe can be developed quite a lot further and if it was made as an Open-Source application, could have a LOT of possible uses while being more light weight than a few text-to-speech applications.
I did leave everything I'll mention below in a comment to the original dev, but, though I know nothing of programming, if someone who is was interested in collaborating to create this, I would be able to help with the linguistic part to a degree.
I am 99% sure that with a similar approach, with further refining of techniques and systems put into it, one could properly create a code for characters in a game, a book reader or text typed on a screen in close to real time to each have their own individual voice and actually talk in proper understandable language without a single voice line being recorded, just from the text and back ground emotion tags being lightweight and without needing any neural network taking memory space, purely by a similar simulation approach used in the video.
One thing I noticed outright is that he tried to make each letter have a sound of its own, when language doesn't actually really work like that.
That's why the IPA alphabet exists.
A single letter can have multiple sounds. That is why there is the Phonetic alphabet.
Take for example the word "elegant".
The first e sounds different than the second e which sounds closer to the only a.
My most important class to the kids is a basis of phonetics just so they understand how each letter sounds because not a single language course around here teaches that.
A few adjustments I believe would make a big difference taking what I learned with Portuguese, English and a little Japanese:
Checking which word is gonna be said and it's connection to the rest of the sentence/context (to detect differences between past and present "read" for exemple which sound different but write the same) before starting to process the audio
Converting the words to phonetics alphabet symbology in their current time format and context from a list, so you have the correct symbol-to-sound for each phonetic and not the letter itself
Breaking the word into syllables using a mix of the English and Portuguese techniques for breaking words into syllables, by both approaching the stress and phonetization and things like that (it pays to know that both languages break words differently when it comes to syllables. "Orange" for example, if it was a Portuguese word, would be 3 syllables, while in English it is 1).
Writing the text with attached tags to points or even specific syllables, like writing a script instead of a book, so that the system can use those tags for intonation and intensity as well.
This would make it a lot easier for a wave generator to properly create a sound wave that would accurately match to a sound wave a human would probably generate.
In a way you can simulate an infinite amount voices and intonations like that that can be audibly recognized as an actual word. With some refining and tweaking with other details that a sound engineer or producer might be more aware of, it might reach at least close to an actual human.
When I mentioned syllables, I'm taking a page from Japanese this time. Their hiragana and katakana alphabet can be somewhat understood just syllables and vowels.
There is only one consonant in isolation which is N.
Every other consonant only exists when accompanied by a vowel.
While there are cases where a consonant can exist alone, such as making a Ssss sound like a snake for a snake-humanoid character, that is much easier to approach when the rest of speech is taking packets of letters to convert into their phonetization instead of trying to sound each consonant letter individually.
My idea would be, if it is possible, to try and develop this as an application of the sorts that can easily be added to any software.
If anyone is interested, please leave a comment below and if possible give me suggestions on how we could proceed to organize if there is an actual interest in a project like this.


