The judge said the camera held still. It had not: measuring viewpoint drift in AI room renders

We build an AI virtual staging tool: a user uploads a photo of a room, an image model restyles it, and automatic checks compare the result with the original before anyone sees it. We described the three gates in an earlier post. This one is about the second gate alone, the one we could not get from a language model.

The image model sometimes moves the camera. Not the furniture: the camera. The render comes back from a step back and to the left, or an angled bedroom shot is straightened to face the bed, or the frame has crept in toward the fireplace. The room in the picture is not the room in the photograph, which is exactly the complaint buyers have about a house that does not look like its listing photos.

Some MLS rule books require an unaltered image with the same view beside an altered one ( the Texas rules, for example ); a render shot from elsewhere cannot satisfy that.

Our vision judge, a model that receives both images and answers a fixed list of yes/no questions, did not see it. Three rounds of hand-verified cases passed every check; one retry passed with every feature on the right wall and the room visibly shot from somewhere else. We gave the camera its own boolean; still blind. We blended the two images 50/50, double-exposure style, and asked whether the architecture doubled; not reliable either. Months later a stronger judge had the opposite problem, calling 13 of 43 in-place renders moved; its camera answers no longer count indoors.

So: can a few hundred lines of plain image maths answer "did the camera move?" better than a model that describes rooms in paragraphs?

Six approaches, A to F in the research scripts, on a hand-labelled set that grew to 24 pairs across two rooms: five with the camera marked as moved, nineteen without.

A, edge correlation in the top strip. Furniture rarely reaches the ceiling, so compare only the top 28% of the frame. Seven iterations later: 3 of 5 shifts caught with no false alarms; the two misses lived mid-frame (an arched window where the photo had three) and never reached the strip.

B, the ceiling line. Trace where wall meets ceiling as a curve, one y per column, by dynamic programming over the vertical brightness gradient, and compare the original's curve with the render's. Clean on the first room. On the second, 8 false flags out of 11 good renders: art over a mantel and dark flush beams pulled the tracked line off the ceiling. Paint fools it.

C and F, block matching on edge maps and a vote-based re-tracker, never separated the set cleanly. E was the double-exposure judge above.

D, a depth profile. Run a monocular depth model on both images. In the upper band of the frame take, per column, how far the far wall reads, and compare the two profiles. Furniture changes the depth model's own normalisation between images, so the comparison is a robust affine fit, a rank correlation and a search for a sideways slide. Glass fools it: windows and mirrors are re-estimated differently each time.

// one number per column: how far the far wall reads in the upper band function farFieldProfile ( depth: Float32Array ): Float32Array { const { med, mad } = bandStats ( depth ); // robust centre and spread const prof = new Float32Array ( W ); for ( let x = 0; x < W; x ++ ) { const col: number [] = []; for ( let y = bandTop; y < bandBottom; y ++ ) col. push (( depth [ y * W + x ] - med ) / mad ); col. sort (( a, b ) => a - b ); prof [ x ] = quantile ( col, FAR_Q ); // low percentile = farthest } return medianFilter ( prof, SMOOTH_WIN ); } Enter fullscreen mode Exit fullscreen mode The two failure modes do not overlap: paint is flat in depth, glass does not move the ceiling line. So the score is the smaller of the two normalised scores, and a render is flagged only when both channels fire:

const comb = Math. min ( rgbScore / T_RGB, depthScore / T_DEPTH ); const verdict = comb > GREY_HI? " shifted ": comb >= GREY_LO? " grey ": " ok "; Enter fullscreen mode Exit fullscreen mode On the 24 pairs: 0 false flags, 5 of 5 shifts caught. The margin was thin: the worst good render scored 0.920, the weakest real shift 1.082. Hence a grey zone, logged and never failed.

The referee runs beside the vision judge on every indoor attempt, retries included; the depth call (about half a cent, three to seven seconds) hides behind the judge's call. "Shifted" counts as a geometry defect, the same class as a missing door, not a styling miss: the attempt is rejected, and when the cheap image model breaks geometry the pipeline escalates to a pricier one instead of rerolling. Everything fails open: if the referee errors or runs out of time, the camera is simply left unchecked and the other gates decide. A flaky referee must never block a paying user.

Fail-open was where the bugs were. The check's 15-second wall used to fail open by resolving past a slow depth call, leaving the prediction running and billing us for a verdict we had abandoned; now it aborts the call. The original's reference had no wall at all, and the depth model is a shared one that scales to zero: a cold start rejected the whole QA stage, and a finished, already-paid render was thrown away as "Generation failed" after 55 seconds.

In September, 8 of 26 live renders lost the camera check because the cold model did not start within 15 seconds (the 215 calls that did start waited at most 2.1 s). The render window has no slack for a sleeping model; the upload does, since a customer spends at least 30 seconds, typically about 60, choosing a style. So the reference is now computed at upload and stored as 1,536 numbers (ceiling curve, glare mask, depth profile); the referee never looks at the original again.

Was the referee too strict? Over 14 days in September, 78 customer renders went out as drafts. The referee had flagged the camera on 29 of them, but the camera was the only failed check on 6, and those six scored 1.33 to 5.40 against a line at 1.08. All seven borderline flags (1.08 to 1.20) sat on renders that had also failed something else. Raising the line to 1.25 would have flipped 7 of 67 flags and un-drafted nothing.

In the lab that month, four full-pipeline runs (345 attempts) produced 25 flags. Checked against the originals, 25 of 25 had really moved, 24 obviously, and none alone: every one also failed a door, window or layout check. The pricier model drifted at the same rate as the cheap one (8 of 104 versus 15 of 192). Camera drift is a symptom, not a class of defect: the model rebuilds the room as a catalogue photo. A wardrobe cut by the frame edge becomes a wall and the camera steps back behind it; an angled bedroom shot is straightened to face the bed; the frame zooms in toward the archway.

Two fixes aimed at the model, both measured, both discarded. A prompt anchor saying the frame is fixed cut camera flags from 10 to 6, but clean renders fell from 15 to 12 and missing doors rose from 11 to 14; reverted. A second premium retry when only the camera had failed: the condition fired in 1 of 39 cells, and the retry gave 0 clean images in 12 throws, the camera returning five times, each with a new defect; not merged.

Then live pairs. Of 36 live drafts with a camera flag, 12 renders still existed (free-plan images expire) and one original; the rest were reconstructed from the stored reference curve and the door and window boxes. By eye: the model had moved the camera in 5 (plus one probable), the referee was wrong in 3, and 3 were arguable.

All three errors were the same error. They scored 1.12 to 1.17, barely over the line, and each room had paint and glass together: a towel rail and a glass shower, a picture shelf and a full-height window. The ceiling-line channel locked onto the decor, the depth channel edged over on the glass, and the minimum rule, built on the assumption that the two deceivers do not meet, met them in one room. The one real soft shift in the set scored 1.10. No line separates 1.10 from 1.12; the calibration rooms simply had neither towel rails nor glass showers.

One blind spot is by construction. A pan in a room with a flat ceiling barely changes the ceiling line's height, so the RGB channel stays quiet and the minimum discards the depth channel's alarm. Of 12 reviewed cases where the judge said moved and the referee said no, 7 were real, five of them vertical bedroom photos straightened to face the bed. One such render in 192 lab cells reached a user clean.

A second turned up in October, on 20 renders of hard amateur photos (close-ups, cluttered rooms). The referee caught 2 of 5 framing failures and missed 3, and in all three the camera had pulled straight back while the model drew in more room. The referee compares where the ceiling line and the far wall sit across the frame, not how large they appear; a pure pull-back changes scale, not position. Of the 15 renders that looked fine it flagged one, and a second look found a real shift there too. A cheap frame-scale comparison caught 2 of the 3 misses in the same set, with false alarms of its own, so it needs its own calibration, in a separate module, before it ships.

The file is frozen; any edit means re-running the calibration. The proposed fix for the near-line errors, cross-checking the door and window boxes before a near-line "shifted" verdict, is written down and not yet done. Until then the referee is right far more often than the model it watches, and wrong in ways we can name.