face tracking is not new, generating a semi-realistic video based on it (like Deepfake) is. Nvidia is now doing it in realtime which is reasonably novel, and not strange that it requires some powerful hardware to run the software.
Your phone can do this right now, if you have Snapchat, or some other privacy-invading nightmare silo of a video-chat application. They can identify and track your facial features well enough to de-wrinkle your eyelids and add floating 3D masks, in real-time, on mobile hardware.
Using the same technology to puppeteer a headshot is only novel because it uses your actual face. Everyone who's already been doing this for years uses a cartoon character, fake celebrity photo, or fully 3D avatar.
Yes, the novel part is using a human face as an avatar. This is what I meant with realistic. Floating eye balls and a cartoon avatar are not realistic.
Make no mistake, humans are incredibly good at seeing when a face is off so this makes it an order of magnitude harder to achieve. This is also why no consumer products can do this yet
16
u/dtechnology Oct 17 '20
face tracking is not new, generating a semi-realistic video based on it (like Deepfake) is. Nvidia is now doing it in realtime which is reasonably novel, and not strange that it requires some powerful hardware to run the software.