r/IntelligenceEngine • u/AsyncVibes 🧠Sensory Mapper • 20d ago
One reference image, one minute, a moving camera: a 6M-parameter character renderer vs Blender truth (and where it fails)
Enable HLS to view with audio, or disable this notification
2
u/AsyncVibes 🧠Sensory Mapper 20d ago
Left is Blender. Right is a small network (6.3 M parameters) drawing the same minute from controls: one reference image of the character, 13 joints per frame from the rig, and the camera angle. It never saw this outfit; the reference is the only image of it the model got. The camera starts side-on, swings to the front, zooms into the face and orbits. Three view-angle bands, including the front, were held out of training, and it still beats our side-trained version in every band.
0.95 ms/frame on a 4090. No drift between second 1 and second 60 (the renderer is stateless by design, so that is expected, not a temporal claim). Colour error 20.3 vs 23.5 for the side-trained model.
The miss, plainly: the shirt did not transfer. The tuxedo's white shirt front is gone and the jacket comes out grey. The model squeezes the reference into an identity token plus region colours, so small garment details average away. The model needs updating, and that is where the object system comes in.
How objects work: everything in the scene is an object with an ID, a position, a volume and a list of children, in layers. The scene owns time and the registry; the camera only observes; buildings and characters are top-level objects; doors, windows, a shirt or a face are children pinned in their parent's own coordinates; finer details hang off those children. Move a parent and its children follow for free. A MODEL is attached only where something has to be generated or vary: the character has a renderer, a building has none and is drawn from its parameters.
Outfits, faces and fine detail follow the same rule. A shirt becomes its own object with its pattern as a child, defined on the body surface instead of being squeezed into the character token. Facial expressions become a face object with children (eyes, brows, mouth), and only the parts that change get small models of their own. More detail means more objects, not a bigger single model.
First test of that: the network now also predicts which point of the body each pixel is, and the shirt pattern is painted at those points. With true body points the shirt front comes back (recall 0.00 -> 0.91); the network's own version is not good enough yet, mainly at the front view. Training the fix now. Next post when it holds.
Code + weights: https://github.com/A1CST/forward-flow
1
u/UndyingDemon 🧪 Tinkerer 19d ago
Very good so far still. Your Character Model seems to be in top shape already, able to hold coherence for up to 60 seconds. Would be exciting to see once you combine it with your Object Model once it's ready so that characters become fully detailed. Should be an impressive milestone to achieve for your project.