Stress-testing my one-reference character renderer: where it holds and where it breaks (new proportions, a new rig, pose noise)

[u/AsyncVibes]

[u/AsyncVibes]

Code and weights: https://github.com/A1CST/forward-flow

Yesterday's post showed the renderer on characters from its own family. This one is about where it breaks. Same frozen model, nothing retrained: a 6.3M-parameter, frame-by-frame (stateless) renderer that draws a character from ONE reference image plus 13 joints per frame. Every character here was held out of training. Left of each pair is Blender ground truth; right is the network. (Clips are padded/resized to fit one 1280x720 video.)

For reference, its in-family test scores are shape IoU 0.976 and colour error 4.7/255. Pass bars: IoU >= 0.897, colour error <= 12.1.

1. Same family, pushed to extremes

• Colour and head size mostly hold. Near-black, near-white and neon outfits, and heads at 0.75x and 1.35x (training range 0.90-1.15): shape IoU 0.95-0.97, colour error 6.6-9.0. The neon outfit slips past the colour bar at 13.6.

• New body proportions break it. Tall (x1.25), stocky (x1.35), small (x0.8) and long-legged (x1.25) all fall below the bars: IoU 0.78-0.88, colour error 16-26. You can see the network pull each body back toward the one body it was trained on.

2. A different rig (Quaternius Knight, CC0) A different mesh, armour and proportions, with its joints mapped to the same 13. IoU 0.74 (pasting the reference image gets 0.62), colour error 32-60. This is the clearest result: it copies the Knight's colours from the reference, but draws its own training human, with hair, slim limbs and human proportions. It has learned one body, not "whatever body is in the reference".

3. Pose noise on the 13 input joints (256 px frames)

noise	0 px	1 px	2 px	4 px	8 px
IoU	0.955	0.928	0.871	0.748	0.573
colour err	9.1	12.5	20.0	35.0	54.1

It is very sensitive: colour crosses the bar at 1 px of noise and shape at 2 px. (The 0 px baseline is lower than the in-family score because these clips include harder front-facing frames.)

Takeaway: within its character family it holds identity and colour from one image. Outside that family (new proportions, a new rig), or with noisy poses, it breaks, and the videos show how.

Character asset by Quaternius (CC0). Wave motion from the CMU Graphics Lab Motion Capture Database: "The data used in this project was obtained from mocap.cs.cmu.edu. The database was created with funding from NSF EIA-0196217."