Write a photograph as I = A ⊙ Sd + R: albedo times shading, plus a residual for highlights. From one image, a grey wall in shadow and a dark wall in full light are the same pixels, so whatever the model cannot attribute to the light ends up in the albedo. Here, "material colour" means the diffuse linear-RGB albedo, not the spectral reflectance.
Figure 1. Existing models under two lights. CRefNet stays similar across the two lights partly because its albedo is pale, and Marigold-App keeps more colour but shifts with the light.
A second photograph of the same scene under different light breaks the tie: the surfaces are the same in both, so anything that changed is lighting. This follows Siamese learning and multi-view consistency in 3D reconstruction, with positive pairs only. We use the Multi-Illumination Dataset (MID), which photographs every scene 25 times with a flash bounced in a different direction, and the pseudo-ground-truth albedo that MIDIntrinsics derives for all 25 frames.
Figure 2. One MID pair. Same camera, same objects, the flash bounced in a different direction. The flash is white, so the light mainly changes in direction and intensity.
Figure 3. CIAI at training time. One shared model, two passes. The thumbnails are outputs of our model.
The model is a frozen DINOv2-L encoder with a DPT decoder, an albedo head and a three-channel shading head (18.5M trainable parameters, 0.148 s per 512×512 image). It is first trained with supervised losses on Hypersim and InteriorVerse, and CIAI is added as a second stage on raw MID pairs. Linv says "agree across lights", and the supervised losses say what that agreement should look like; agreement alone could also be satisfied by the same wrong albedo in both frames.
The frozen encoder tends to wash out the albedo colour, so the model also has a colour path: an RGB skip into the albedo head and a chroma loss on albedo. This is a design choice of the model, not part of CIAI, and the skip can pass the light colour of the input into the albedo as a colour cast.
A grey, constant albedo is perfectly stable, so we always report stability next to accuracy.
Figure 4. What each MID score catches. Cmat and Castrel ask whether a material stays put across lights; Chromaerr asks whether that colour is correct. A grey albedo can score well on the first two, so they are always read next to Chromaerr and MAW.
| Model | Cmat ↓ | Castrel ↓ | MID chroma err ↓ | MAW ΔE ↓ |
|---|---|---|---|---|
| Ours | 0.157[.141–.174] | 0.425[.360–.491] | 0.129[.109–.150] | 4.16 |
| CD-IID | 0.190[.172–.210] | 0.306[.262–.355] | 0.090[.073–.110] | 3.37reported |
| RGB→X | 0.128[.115–.141] | 0.338[.273–.409] | 0.203[.170–.239] | – |
| CRefNet | 0.151[.138–.164] | 0.355[.309–.406] | 0.201[.165–.241] | 3.97 |
| Marigold-App | 0.193[.173–.214] | 0.355[.309–.406] | 0.195[.152–.245] | 3.78 |
| Marigold-Light | 0.546[.462–.645] | 0.392[.333–.459] | 0.154[.124–.187] | 4.21 |
| Ordinal Shading | 0.252[.229–.279] | 0.549[.463–.637] | 0.148[.130–.166] | 6.88 |
Table 1. MID held-out split, 30 scenes, raw input, one harness for every method; brackets are 95% bootstrap intervals over scenes. MAW uses the official scorer, except CD-IID, whose value is from its paper.
In paired per-scene tests, our model is more luminance-stable than CD-IID, both Marigold variants and Ordinal Shading, level with CRefNet and behind RGB→X. However, CD-IID, which predicts the shading colour in a separate stage, is better on all colour measures. A low drift can also come from a pale albedo, as CRefNet shows in Figure 5.
Figure 5. One held-out scene, three flash directions, seven methods. The red and blue boxes track the same two regions in every panel, so any change between rows comes from the model.
Pick a model and step through four held-out scenes. Every prediction is a real checkpoint output.
| CIAI off | CIAI on | ||
|---|---|---|---|
| Colour path off | Row 1Cmat 0.250 | → | Row 2Cmat 0.180 |
| Colour path on | Row 3Cmat 0.259 | → | Row 4 (ours)Cmat 0.157 |
Table 2. One shared checkpoint, same data and steps. Each arrow changes only the CIAI losses.
Figure 6. Reflective surfaces. The mirror sphere from the teaser: its reflections change with the light in the input and in our albedo.
The next model splits the shading into a grey part and a colour part, so that a change in light colour has a place to go other than the albedo. It is planned and not trained yet.
Figure 7. Planned pipeline. The output panels are ground-truth targets from a rendered 3D-Front scene (S = I / A), not predictions. Light-grey pixels are masked, and chroma is shown at constant luminance.
@misc{tran2026ciai, title = {Cross-Illumination Albedo Invariance: Luminance-Stable Intrinsic Decomposition from Real Multi-Illumination Photographs}, author = {Tran, Minh Khang}, year = {2026}, note = {Project report, revised from the author's MSc thesis (Erasmus Mundus Joint Master in Computational Colour and Spectral Imaging, NTNU)}, url = {https://github.com/tmkhang1999/CIAI} }