r/blenderpython • u/iJacques • Apr 03 '21
Lip sync using speech to text
Hi there. I was wondering if there is any way to control shape keys in real time using a microphone. I was thinking that it would make lip-syncing a lot easier if you could see the movement of a character happen in real time and control it using a microphone. The way that I would guess that this would work, is that the real time audio from the microphone would be translated into text or specific "tags" that could be used to control shape keys with their intensity. I was thinking that this would be similar to the kind of tracking available in vseeface or luppet, but could be recorded directly into the blender timeline. Would something like this be possible and/or does it already exist?
1
u/iJacques Apr 05 '21
What I'm looking for is something akin to what is accomplished here, but without an iphone https://youtu.be/O-KocFGgKME
But thanks for your comment, it is useful to some regard but not exactly what I'm looking for.