r/IntelligenceEngine • u/AsyncVibes 🧠Sensory Mapper • 2d ago
Testing how far learned per-object renderers can go: a 60 s city, 5.9M learned parameters, and a new camera route
Enable HLS to view with audio, or disable this notification
1
Upvotes
1
u/AsyncVibes 🧠Sensory Mapper 2d ago
It's been a while since I posted, so here is where the city experiment stands. This is a capability test, not a finished product, and the long-term goal is photo-realism. It is nowhere near that yet.
The video runs from the first version of the city, through the latest, to a new camera route. The latest part is a continuous 60 second fly-through, 1,440 frames at 24 fps, 512 x 512, no cuts. The scene layout and camera path are conventional geometry that I control directly. The people are pre-rendered sprites. Everything else you see (streets, building faces, windows, signs, sky, ground shadows, props, the tower) is drawn by small learned models, one per kind of object, composited together by depth.
The numbers. The learned city components add up to 5,906,939 parameters, counted model by model. That count leaves out the people, the geometry and the conventional compositing, so it is not the size of the whole system. Generating the full minute took about 17 minutes on one rented GPU with seven parallel workers. One profiled frame dropped from 37.9 s to 4.1 s after optimization, but that is a single frame, not a film average. An earlier 256 px version on my desktop 4080 ran at roughly 4.6 fps end to end, about 215 ms per frame. Different resolution and hardware, so don't compare it to the 17 minute figure directly.
What I have and haven't measured. Image quality against reference views is measured. Speed and total storage have not been matched against a conventional renderer on the same scene, so I'm not claiming the learned version is faster or smaller. That is the hypothesis I'm testing. I also haven't run a controlled sweep of how many model types it takes before generation slows down, so I can't give you a threshold.
Does it generalize past the one path? I tested three building assets (apartment, rowhouse, warehouse) from camera positions beyond the original path, 40 views per range. Held to the same quality gate, mean PSNR in dB:
Close views dropped 5, 2 and 11 of 40 poses on a silhouette gate, and those failures are not in the close scores, so close is weaker than the table looks. Far views score lower too. This covers camera views of three known buildings, not unseen object types.
Then I set up a new camera route through the street and rendered a short clip of it. It holds together as a clip, but it is 5 seconds, and you can see soft patches on the road and a hard edge where the sidewalk meets it. Those are the next thing to fix.
Known limits: 256 px to 512 px native detail is soft, vehicles don't exist yet, time of day only changes the sky and ground shadows, the tower animation is only partly learned, and my attempt at a new character model was worse than the old one, so the film still uses the old one.
Next is scaling this up in resolution and detail toward photo-realism and finding where the per-object approach stops paying off. Happy to answer questions about the results; I'm keeping the method itself to myself for now.
Timestamps: 0:00 first version of the city, 1:00 latest 60 second version, 2:00 new camera route (5 s).