Khang Tran · Research   Code
Research project  ·  2026  ·  started as an MSc thesis (COSI / NTNU)

Cross-Illumination Albedo Invariance Luminance-stable intrinsic decomposition from real multi-illumination photographs

Minh Khang Tran1,2,3 Seyed Ali Amirshahi1 Luis Gomez Robledo4 Sezer Karaoglu2,3 Theo Gevers2,3
1 NTNU Gjøvik 2 University of Amsterdam 3 3DUniversum 4 University of Granada
Top row: one indoor scene photographed four times with a flash bounced in a different direction each time, so brightness and shadows change. Bottom row: the albedo our model recovers from each of those four photographs.

One scene photographed with the flash bounced in four directions (top), and the albedo our model predicts from each photograph separately (bottom). The surfaces do not change between the shots, so the predicted albedo should not change either.

TL;DR

Train on two lights, predict from one

Contributions
  1. A paired training strategy on real MID photographs with pseudo-ground-truth albedo, added as a second training stage.
  2. A matched ablation that isolates the effect of CIAI on luminance stability and on albedo accuracy.
  3. An evaluation that pairs stability with accuracy, because a grey, constant albedo is perfectly stable.
Method

Cross-Illumination Albedo Invariance

Write a photograph as I = A ⊙ Sd + R: albedo times shading, plus a residual for highlights. From one image, a grey wall in shadow and a dark wall in full light are the same pixels, so whatever the model cannot attribute to the light ends up in the albedo. Here, "material colour" means the diffuse linear-RGB albedo, not the spectral reflectance.

One MID scene under two lighting conditions, with the albedo predicted by CRefNet and by Marigold-Appearance for each. CRefNet's albedo is pale; Marigold's changes between the two lights.

Figure 1. Existing models under two lights. CRefNet stays similar across the two lights partly because its albedo is pale, and Marigold-App keeps more colour but shifts with the light.

A second photograph of the same scene under different light breaks the tie: the surfaces are the same in both, so anything that changed is lighting. This follows Siamese learning and multi-view consistency in 3D reconstruction, with positive pairs only. We use the Multi-Illumination Dataset (MID), which photographs every scene 25 times with a flash bounced in a different direction, and the pseudo-ground-truth albedo that MIDIntrinsics derives for all 25 frames.

The same table of objects photographed with the flash bounced in two directions, with a zoomed crop of a book cover under each. Brightness and shadows change between the two frames; the light colour changes only slightly.

Figure 2. One MID pair. Same camera, same objects, the flash bounced in a different direction. The flash is white, so the light mainly changes in direction and intensity.

Diagram: two photographs of the same scene with the flash in different directions pass through one shared model. The two predicted albedos are tied by the invariance loss; the two predicted shadings are tied by the explanation loss on the luminance ratio.

Figure 3. CIAI at training time. One shared model, two passes. The thumbnails are outputs of our model.

Linv albedo invariance
the pair must agree
Masked L1 between the two RGB albedos over pixels that are valid in both frames (not clipped, not near black).
Lexpl shading explanation
the shading must explain the change
The log luminance ratio of the two shadings must equal that of the two images. It rules out a flat albedo, and it sees brightness only.

The model is a frozen DINOv2-L encoder with a DPT decoder, an albedo head and a three-channel shading head (18.5M trainable parameters, 0.148 s per 512×512 image). It is first trained with supervised losses on Hypersim and InteriorVerse, and CIAI is added as a second stage on raw MID pairs. Linv says "agree across lights", and the supervised losses say what that agreement should look like; agreement alone could also be satisfied by the same wrong albedo in both frames.

The frozen encoder tends to wash out the albedo colour, so the model also has a colour path: an RGB skip into the albedo head and a chroma loss on albedo. This is a design choice of the model, not part of CIAI, and the skip can pass the light colour of the input into the albedo as a colour cast.

Results

Stability next to accuracy

A grey, constant albedo is perfectly stable, so we always report stability next to accuracy.

Three small diagrams. C_mat: four bars of equal height for a stable material, unequal bars for an unstable one. Cast_rel: chroma points clustered for a stable material, sliding along the lamp colour for a drifting one. Chroma_err: prediction close to the reference colour, or far from it.

Figure 4. What each MID score catches. Cmat and Castrel ask whether a material stays put across lights; Chromaerr asks whether that colour is correct. A grey albedo can score well on the first two, so they are always read next to Chromaerr and MAW.

Model Cmat ↓ Castrel ↓ MID chroma err ↓ MAW ΔE ↓
Ours 0.157[.141–.174] 0.425[.360–.491] 0.129[.109–.150] 4.16
CD-IID 0.190[.172–.210] 0.306[.262–.355] 0.090[.073–.110] 3.37reported
RGB→X 0.128[.115–.141] 0.338[.273–.409] 0.203[.170–.239] –
CRefNet 0.151[.138–.164] 0.355[.309–.406] 0.201[.165–.241] 3.97
Marigold-App 0.193[.173–.214] 0.355[.309–.406] 0.195[.152–.245] 3.78
Marigold-Light 0.546[.462–.645] 0.392[.333–.459] 0.154[.124–.187] 4.21
Ordinal Shading 0.252[.229–.279] 0.549[.463–.637] 0.148[.130–.166] 6.88
Best in column Clear outlier

Table 1. MID held-out split, 30 scenes, raw input, one harness for every method; brackets are 95% bootstrap intervals over scenes. MAW uses the official scorer, except CD-IID, whose value is from its paper.

In paired per-scene tests, our model is more luminance-stable than CD-IID, both Marigold variants and Ordinal Shading, level with CRefNet and behind RGB→X. However, CD-IID, which predicts the shading colour in a separate stage, is better on all colour measures. A low drift can also come from a pale albedo, as CRefNet shows in Figure 5.

Grid comparing input, pseudo-ground-truth albedo, and predictions from seven methods across three flash directions of the same scene. Two regions are tracked with coloured boxes across every panel: a mirror-finish calibration sphere in red, and a dark jacket in blue.

Figure 5. One held-out scene, three flash directions, seven methods. The red and blue boxes track the same two regions in every panel, so any change between rows comes from the model.

Try it yourself

Pick a model and step through four held-out scenes. Every prediction is a real checkpoint output.

Input Input photograph
Predicted albedo

What the ablation isolates

CIAI offCIAI on
Colour path offRow 1Cmat 0.250→Row 2Cmat 0.180
Colour path onRow 3Cmat 0.259→Row 4 (ours)Cmat 0.157

Table 2. One shared checkpoint, same data and steps. Each arrow changes only the CIAI losses.

  1. CIAI improves luminance stability. Cmat drops by 28% and 39%, while hue drift (Castrel) improves only by 6% and 4%.
  2. CIAI improves accuracy where the input carries its lighting. On raw ARAP renderings, albedo LMSE drops by 14% and 7%; after white balance the change is small. ARAP is never used in training.
  3. CIAI does not improve MAW. ΔE goes from 4.43 to 4.63 and from 4.12 to 4.16.
Limitations

What CIAI does not fix

A mirror sphere under four flash directions (top) and our predicted albedo for each (bottom); the reflections change between the four albedos.

Figure 6. Reflective surfaces. The mirror sphere from the teaser: its reflections change with the light in the input and in our albedo.

Next steps

From luminance to colour

The next model splits the shading into a grey part and a colour part, so that a change in light colour has a place to go other than the albedo. It is planned and not trained yet.

Pipeline diagram. Two photographs of one rendered living room, one under an orange light and one under a green light, go through a shared model. Each pass outputs an albedo, a grey shading and a shading chroma. The two albedos are tied by L_inv, the two grey shadings by L_lum_expl, and the two chroma maps by L_chr_expl, which applies only when the pair's illuminant gap is at least 0.08.

Figure 7. Planned pipeline. The output panels are ground-truth targets from a rendered 3D-Front scene (S = I / A), not predictions. Light-grey pixels are masked, and chroma is shown at constant luminance.

  1. Cross-architecture test. Train with and without CIAI on a CNN, a fine-tuned ViT and a single-step diffusion model, with the same data and three seeds.
  2. Model. Predict I = A ⊙ (Slum · C) + R, and feed the albedo head the corrected image I / (Slum · C) instead of raw RGB.
  3. Loss. Add a chroma explanation loss on C, used only on pairs whose light colour really changes.
  4. Data. Rendered coloured-light pairs from 3D-FRONT with exact albedo, and real coloured-light pairs such as LSMI.

Citation

@misc{tran2026ciai,
  title  = {Cross-Illumination Albedo Invariance: Luminance-Stable Intrinsic
            Decomposition from Real Multi-Illumination Photographs},
  author = {Tran, Minh Khang},
  year   = {2026},
  note   = {Project report, revised from the author's MSc thesis (Erasmus Mundus
            Joint Master in Computational Colour and Spectral Imaging, NTNU)},
  url    = {https://github.com/tmkhang1999/CIAI}
}