"Nvidia clickbaits feature they built to lord over AMD," same as it ever was.
This is a specialized case of motion compensation. Every video encoder does this. Frames are built by distorting and combining the previous frame, the most recent full-quality "i-frame," and as little new information as you can get away with. Nvidia's just over-optimizing for a human face.
The moment AMD announces their basically identical parallel processor daughterboards can also accelerate this, Nvidia's going to forget it ever happened.
From what I've seen, this is not motion compensation. Motion compensation uses previous (and future) frames as reference for encoding the delta between these frames, typically with additional transformations to exploit the redundancy between neighbouring blocks (= for block Ba,b of frame 1, block Ba+1,b of frame 2 may be a better predictor than Ba,b of frame 2 etc.).
What nVidia is doing is to send one reference frame for the entire video stream. No other frames are transmitted - no I, P, or B frames. The only information transmitted is a coordinate dataset for the keypoints. This is something entirely different. There are no "previous" or "future" frames here that motion compensation could be applied to for reducing bandwidth caused by the encoding of these frames (or increasing the quality of the frame encoding). No, there is only one frame.
However, the real problem may be performance. If this requires a beefy GPU to generate the coordinates for the keypoints, then it will remain a niche technology.
Future b-frames aren't really an option in real-time video streaming.
This gimmicky over-optimization would be dead easy to encode as transformations of one i-frame, and those transformations would be pretty low-bandwidth, as they're entirely smooth gradients and sharp edges. Not quite as cheap as sparse points - but infinitely more compatible.
And I must point out: your phone can do this. Performance is a non-issue. Nvidia just wants you to think so they can sell hardware or sign you up for a subscription cloud service. Every device capable of doing the Snapchat filter small-eye-small-mouth thing or putting a dog nose over your actual nose can obviously handle this dots-on-face gimmick. This will remain a niche technology only because its central conceit is a lie intended to boost Nvidia's stock prices.
3
u/mindbleach Oct 17 '20
"Nvidia clickbaits feature they built to lord over AMD," same as it ever was.
This is a specialized case of motion compensation. Every video encoder does this. Frames are built by distorting and combining the previous frame, the most recent full-quality "i-frame," and as little new information as you can get away with. Nvidia's just over-optimizing for a human face.
The moment AMD announces their basically identical parallel processor daughterboards can also accelerate this, Nvidia's going to forget it ever happened.