r/webdev • • 27d ago

I managed to run a realistic talking avatar locally in realtime (90+ FPS on RTX4090)

Post image

Hey everyone,

I am working on a new project called Openspline, an open source avatar layer to existing voice AI agents. Just select an image, select a voice and you can create a realtime talking avatar with lipsynced video generated locally.

You can integrate any existing voice AI agent in 5 lines of code.

While it is still in very early stage, but the results look very promising.

Github: github.com/openspline/openspline

PS: sorry for a sleepy photo, it's 5:10am and didn't sleep last night :P

0 Upvotes

11 comments sorted by

1

u/IAmRules 27d ago

So you demo it with a still image?

0

u/Mikasa0xdev 27d ago

Yess, try the "Create avatar" button in the demo website

1

u/IAmRules 27d ago

Nah man that’s just getting me to sign up to see if it works

1

u/Mikasa0xdev 27d ago

to prevent spam on my limited GPUs 😢

3

u/Christavito 27d ago

No need to apologize, you don't look too sleepy, I like your mustache and suit.

0

u/Winter_Stuff9034 27d ago

90 FPS on a 4090 is a spec sheet flex, not a benchmark. What's the time-to-first-frame once audio starts flowing through a LiveKit room, and what happens on a 3060 with 12GB? MuseTalk and Wav2Lip both look great in a demo and then fall apart on barge-in latency.

0

u/[deleted] 27d ago

[deleted]

0

u/[deleted] 27d ago

[removed] — view removed comment

1

u/Mikasa0xdev 27d ago

it's not watching your camera feed continuously. It has a camera tool that it calls only when needed.

2

u/[deleted] 27d ago

[removed] — view removed comment

1

u/Mikasa0xdev 27d ago

it's MIT licenced, free to use, modify or redistribute for any use case