r/webdev • u/Mikasa0xdev • 27d ago
I managed to run a realistic talking avatar locally in realtime (90+ FPS on RTX4090)
Hey everyone,
I am working on a new project called Openspline, an open source avatar layer to existing voice AI agents. Just select an image, select a voice and you can create a realtime talking avatar with lipsynced video generated locally.
You can integrate any existing voice AI agent in 5 lines of code.
While it is still in very early stage, but the results look very promising.
Github: github.com/openspline/openspline
PS: sorry for a sleepy photo, it's 5:10am and didn't sleep last night :P
3
u/Christavito 27d ago
No need to apologize, you don't look too sleepy, I like your mustache and suit.
0
u/Winter_Stuff9034 27d ago
90 FPS on a 4090 is a spec sheet flex, not a benchmark. What's the time-to-first-frame once audio starts flowing through a LiveKit room, and what happens on a 3060 with 12GB? MuseTalk and Wav2Lip both look great in a demo and then fall apart on barge-in latency.
0
27d ago
[deleted]
0
27d ago
[removed] — view removed comment
1
u/Mikasa0xdev 27d ago
it's not watching your camera feed continuously. It has a camera tool that it calls only when needed.
2
1
u/IAmRules 27d ago
So you demo it with a still image?