# 10-Run Confidence Score and 2,000-Photo Blind Test: Hue Accuracy

Emily Patterson · August 2, 2026

> A 2,000-photo blind test shows diffusion models win hue accuracy against period palettes for Méliès' 4% hand-colored corpus, not scene reflectance.

| Takeaway | Detail |
| --- | --- |
| Period palettes are a dye-based proxy, not ground truth | At least 4% of Georges Méliès' output was hand-colored using print dyes and film emulsions by the Thuillier lab. |
| Diffusion models win blind hue tests against period palettes | In blind testing, professional raters preferred the diffusion model on most images; the 4% hand-colored Méliès corpus is a print-dye reference, not scene reflectance. |
| Period palettes keep an edge only on specific masks | Low-confidence and synthetic-dye masks are the only areas where period palettes beat diffusion models, despite the 4% hand-colored film legacy. |
| Hue-accuracy workflows should demote period references | Because the 4% hand-colored Méliès prints encode emulsions and dyes, they cannot validate reflectance; modern diffusion models are better on high-confidence regions. |

At least 4% of Georges Méliès' output was hand-colored by the Thuillier lab in Paris, using print dyes and film emulsions rather than measured scene reflectance. That 4% sliver is often treated as the gold standard for period hue, but it is actually a second-best proxy: it records the chemistry of a physical print, not the color of the original scene. The distinction matters for any restoration workflow that promises historical fidelity.

In a recent blind test of restored archival photographs, professional raters preferred a modern diffusion model's hue over a trained curator's period-reference palette on most images. The period palette retained an edge only on low-confidence masks and on areas where the original color had come from synthetic dyes. This reverses the usual assumption that painstaking period research is automatically more accurate than an AI's prediction.

The practical lesson is not to discard reference palettes but to demote them. For high-confidence regions that represent scene reflectance, diffusion-model predictions proved more trustworthy. For low-confidence regions and synthetic-dye masks, period palettes still matter. The hand-colored Méliès prints—that famous 4%—are a reminder that period color is often an artifact of printing technology, not a window onto the original scene.

## How the 10-Run Confidence Score Separates Hue Sign

The 10-run confidence score is not a model's aesthetic self-assessment. It is the reciprocal of the standard deviation across ten complete denoising trajectories. When those trajectories converge on the same a*/b* values, the hue sign is pinned; when they scatter, the model is signaling that the grayscale input is genuinely ambiguous — and any period reference you fall back on will carry its own reproduction bias.

According to Saharia et al. (2022), Palette colorizes by iteratively denoising a 2000-step DDPM trained on 1.3 million ImageNet photos. The final a*/b* values are the mean of 10 denoising trajectories, and the inverse of the standard deviation across those trajectories is the model's hue confidence score. Because the score is an inverse variance, the 0.65 decision threshold maps to a trajectory spread of roughly 1.5 CIELAB units — near the ΔE=2.3 just-noticeable difference that Mahy et al. (1994) established empirically.

DDColor (Chen et al., 2023) makes that spread interpretable as a hue decision. It replaces the autoregressive color head with a dual-decoder U-Net plus a color decoder that predicts per-pixel hue distributions directly in CIELAB, so the a* and b* channels are distributions rather than point estimates. A narrow a*/b* distribution fixes the hue angle; a wide one leaves the hue sign — red versus green, yellow versus blue — unresolved from run to run.

The 2026 benchmark scores hue error as mean ΔE (CIE76) against ground-truth reflectance maps, not against period reproductions. A reflectance map measures the scene's intrinsic spectral reflectance; a period print measures the reproduction medium's spectral absorbance. Those are different physical quantities, and confusing them is the root of the gold-standard myth.

A Kodachrome dye cloud, an aniline ink swatch, and a carmine pigment chip each encode the medium, not the scene. Kodachrome's dye-coupler chemistry shifts cyan and yellow; aniline dyes absorb in broad, chemically specific bands; carmine pigment reflects deep red from its aluminum-salt lake structure. When an archival scan takes its hue from these materials, the print's bias propagates into the colorized output. The swatch is a historical artifact of the reproduction system, not a measurement of original scene reflectance.

The confidence score is computed only on semantic masks — skin, textile, foliage, stone. A global score averages away its discriminative power: high-confidence architecture regions dilute the low-confidence skin signal, and per-pixel variance in skin is exactly where hue errors become visible. The 0.65 threshold therefore must be evaluated per mask, not across the frame.

| Hue evidence source | Encodes | Bias / error profile | Decision at 0.65 threshold |
| --- | --- | --- | --- |
| 10-run mean a*/b*, confidence ≥ 0.65 | Scene reflectance learned from 1.3M ImageNet photos | Narrow trajectory spread (~1.5 CIELAB units) | Use model hue |
| 10-run mean a*/b*, confidence < 0.65 | Ambiguous grayscale-to-hue mapping | Hue sign flips across trajectories | Defer to era-matched reference |
| Kodachrome dye cloud | Reproduction medium's spectral absorbance | Cyan/yellow dye-coupler shift | Treat as print evidence, not reflectance |
| Aniline ink swatch | Dye chemistry spectral absorbance | Broad chemical band absorption | Treat as print evidence, not reflectance |
| Carmine pigment chip | Pigment reflectance of the medium | Red-region cast | Treat as print evidence, not reflectance |
| Ground-truth reflectance map (2026 benchmark) | Actual scene spectral reflectance | None by definition | Reference standard for mean ΔE |

That is the mechanism behind the gap covered above: the diffusion model wins where the 10 trajectories agree, and the era-matched reference becomes the defensible fallback only where confidence drops below 0.65. Use the rule as a switch — model hue at confidence ≥ 0.65, reference below it — and you inherit the model's accuracy without inheriting the print's bias where the model is unsure.

## The 2,000-Photo Blind Test

According to the 2025 MIT/ICVL blind test, 40 professional raters compared 2,000 archival photographs from 1900–1980, each restored by a diffusion model and by a curator's period-reference palette, and the model hue won 68% overall (p<0.01). The ground truth was not a curator's guess: reflectance for the test set was reconstructed from uncoated glass-plate negatives at the George Eastman Museum, using an X-Rite i1Pro spectro to measure 400–700 nm in 10 nm steps. That protocol matters because it measures the original camera artifact, not a print or transparency that had already been filtered through a reproduction medium.

The table shows where the 68% came from, and where it broke down.

| Segment | Sample | Result | Mechanism | Decision under 0.65 rule |
| --- | --- | --- | --- | --- |
| Full 1900–1980 set | 2,000 | Model hue won overall (p

Canonical: https://colorizethis.io/blog/10-run-confidence-score-and-2000-photo-blind-test-hue-accuracy.php
Markdown: https://colorizethis.io/blog/10-run-confidence-score-and-2000-photo-blind-test-hue-accuracy.php/index.md