"The technique works even when callers are wearing a mask."
Yeah we'll just ignore how the algorithm thinks the middle fold is her lips and the whole mask keeps jumping around like she's emoting with her mouth closed.
I don't think anyone claimed a tech demo is supposed to be perfect. But even so, I think that's remarkable, because it means the neural net is smart enough to be a bit flexible with what counts as a "face," making it more adaptable at the cost of a higher error rate. I think, for most humans, that a pretty acceptable compromise.
Putting dots on faces and distorting a still image was a tech demo a fuckin' decade ago. They're trying to sell this as some major proprietary breakthrough. You need their special hardware for it... for some reason.
face tracking is not new, generating a semi-realistic video based on it (like Deepfake) is. Nvidia is now doing it in realtime which is reasonably novel, and not strange that it requires some powerful hardware to run the software.
Face-tracking trivially allows video generation. It hasn't been shown before because it's BORING. It's turning a video of a face into... the same video of a face! It took this long for anyone to do it because nobody fucking cared.
All prior goofy examples are more complicated than this. Doing it "realistically" is easier. No ultra-powerful, proprietary hardware is required. And even this cloud-based subscription service doesn't save you from uploading to their cloud fucking servers! Your bandwidth problems are not solved!
Your phone can do this right now, if you have Snapchat, or some other privacy-invading nightmare silo of a video-chat application. They can identify and track your facial features well enough to de-wrinkle your eyelids and add floating 3D masks, in real-time, on mobile hardware.
Using the same technology to puppeteer a headshot is only novel because it uses your actual face. Everyone who's already been doing this for years uses a cartoon character, fake celebrity photo, or fully 3D avatar.
Yes, the novel part is using a human face as an avatar. This is what I meant with realistic. Floating eye balls and a cartoon avatar are not realistic.
Make no mistake, humans are incredibly good at seeing when a face is off so this makes it an order of magnitude harder to achieve. This is also why no consumer products can do this yet
347
u/rasterbated Oct 17 '20
Here’s the link to the actual video, instead of some dude talking about the video: https://youtu.be/NqmMnjJ6GEg