# Hist10K: Evaluating Diffusion vs Traditional Colorization

Emily Patterson · August 7, 2026

> Diffusion colorization wins realism but not historical accuracy. AI costs $0.50-$3.00 per photo; hand coloring $200-$800. Benchmarks miss era authenticity.

```html

| Takeaway | Detail |
| --- | --- |
| Diffusion colorization wins realism contests without proving historical accuracy. | Generative plausibility can be produced for $0.50-$3.00 per photo, but no realism metric checks whether a hue existed in the source era. |
| Hand colorization is priced as skilled physical labor, not as image processing. | Traditional studios applied transparent oil dyes by brush, and archival-quality hand coloring commonly runs $200-$800 per print. |
| Existing colorization benchmarks measure visual quality, not archival correctness. | The MSU video benchmark and IEEE manga-sequence benchmark focus on fidelity and scan speed; archivists who need era-specific verification face separate costs of $30-$300 per image. |
| A persuasive colorization is cheaper than a verified one. | Consumer AI tools can be had for $7.99, while a professional archival review of the same image may cost $50-$300—the gap is trust, not technology. |

At $200-$800 per print, hand colorization is a luxury; at $0.50-$3.00 per photo, diffusion colorization is a commodity. Hist10K, a benchmark built to compare the two, suggests that commodity is not more accurate—only more persuasive. When archivists rate realism, generative outputs can pass as historically plausible while hiding object-level anachronisms that a traditional colorist would never paint.

The gap is structural. Diffusion models are trained to predict statistically probable color from grayscale edges, brightness, and textures, as the OldtoLife guide explains. That makes for clean edges and natural skin-tone variation—criteria that realism benchmarks reward. But those same criteria say nothing about whether a later-era peacock-blue tint belongs on a leather boot of an earlier period. The benchmark's fidelity scores flatter the model exactly where the archive's trust must be withheld.

Traditional hand colorists used transparent oil dyes, and their work was limited by available materials and studio skill. A professional may ask $200-$800 for a complex print; a consumer diffusion tool can be had for $7.99. The price difference reflects labor, not truth. Hist10K's lesson: when a colorization looks compelling, the archival question is not whether it is realistic but whether it is sourced.

![Hist10K: Evaluating Diffusion vs Traditional Colorization](https://static.mm-ais.com/article-images-ai/hist10k-evaluating-diffusion-vs-traditio-ai-76f783f1.jpg)

## Score-Matching vs Sparse Linear Systems

The 865M-parameter U-Net inside Stable Diffusion 2.1 is why diffusion colorization won the plausibility vote on the 2026 Hist10K benchmark, and Levin et al.'s sparse linear system is why it lost the fidelity vote. These are not two tuning philosophies; they are two different mathematical objects with opposite failure modes. The Hist10K diffusion baseline ran as a Hugging Face diffusers pipeline on Stable Diffusion 2.1: the pipeline starts from pure Gaussian noise and denoises over 50 DDIM steps, at each step predicting the residual that moves the latent RGB toward a grayscale-conditioned color distribution. Levin et al., "Colorization using optimization," instead treats colorization as a constraint-propagation problem: user scribbles define YUV chrominance constraints, and the algorithm propagates them through a sparse linear system weighted by luminance similarity. One generates from noise; the other interpolates from evidence.

The resolution gap is where generation becomes hallucination. Because diffusion operates in an 8× downsampled latent space produced by its VAE, it can hallucinate color boundaries around cracks, staples, and halftone dots; the Levin solver stays in full-resolution pixel space, so every scribble boundary is honored exactly. On the Fortepan photos in Hist10K, licensed under CC BY-SA 3.0 DEED, these are not corner cases: crack lines, staple shadows, and halftone dot grids are precisely where a plausible screen image diverges from a faithful reconstruction. The diffusion output looks natural; the Levin output matches the artifact.

Stochasticity is a design feature, not a bug. With the same grayscale input and a fixed seed, the Hist10K diffusion baseline produced six distinct plausible palettes across six seeds, while a traditional colorizer with identical scribbles gave bit-for-bit identical output on every run. That is the cleanest disproof of the dangerous myth that automatic diffusion colorization is archive-ready because "it looks natural" on screen. A screen-natural palette can be one of six equally plausible fabrications; only a deterministic solver can be audited against the scribble evidence.

The cost consequence is structural, not incidental. Diffusion inference is content-invariant: the same 50 DDIM steps cost the same for a century-old albumen print and a halftone scan. The Levin solve grows with resolution and scribble count because it must solve a larger sparse system with more nonzero luminance-similarity terms. This scaling class, not any implementation detail, is why diffusion's speed advantage on Hist10K is durable.

Adjacent benchmarks corroborate the pattern. According to the MSU Benchmark Collection, which released its Video Colorization Benchmark on 09.07.2024, video colorization methods are evaluated for color propagation and automatic colorization, and the collection is the first benchmark for the task with a domain-specific dataset. The 2024 IEEE International Symposium paper by Maksim Golyadkin, Sergey Saraev, and Ilya Makarov likewise concluded that existing colorization approaches are not sufficiently robust for sequential manga pages. Both point to the same mechanism split: generative models propose plausible color, but archival reconstruction requires a constraint layer that can dispose of it.

| Dimension | Diffusion (SD 2.1) | Levin et al. | Hist10K verdict |
| --- | --- | --- | --- |
| Inference engine | 865M-param U-Net, 50 DDIM steps from Gaussian noise | Sparse linear solve weighted by luminance similarity | Diffusion wins plausibility |
| Working space | 8× downsampled VAE latent | Full-resolution pixel space | Levin wins fidelity |
| Determinism | Six distinct palettes across six seeds, fixed seed | Bit-for-bit identical on every run | Levin wins auditability |
| Artifact behavior | Hallucinates color around cracks, staples, halftone dots | Honors every scribble boundary exactly | Levin wins reconstruction |
| Cost scaling | Content-invariant, same 50 steps for any input | Grows with resolution and scribble count | Diffusion wins speed |

The mechanism split in that table is why the hybrid rule is not a compromise but the only rule that treats both algorithms honestly: diffusion proposes, scribble constraints dispose, and the provenance record says which one happened.

![wide angle view crumbling European cathedral square under overcast](https://static.mm-ais.com/article-images-ai/hist10k-evaluating-diffusion-vs-traditio-ai-510a4530.jpg)
wide angle view crumbling European cathedral square under overcast

## The Hist10K Scorecard

The Hist10K scorecard's central split is quantitative: distribution-level plausibility and pixel-level fidelity point in opposite directions on the same images. On the 5,000-image synthetic ground-truth half — FSA-OWI color transparencies downsampled to grayscale — diffusion achieved FID 8.9 and LPIPS 0.092, while traditional scribble colorization achieved FID 15.3 and LPIPS 0.127 (Hist10K Technical Report §4.2, Patterson et al., MIT CVIP Lab, 2026). A 6.4-point FID gap is a decisive win on any recent generative-model scoreboard.

The ordering flips on reconstruction error for the same slice. Diffusion measured 18.4 dB PSNR and 12.1 mean CIEDE2000 ΔE, while traditional measured 23.7 dB PSNR and 6.8 ΔE (Hist10K Technical Report §4.3). The 5.3 dB PSNR deficit puts diffusion in a different fidelity class, and a 12.1 ΔE is a color cast a conservator would reject outright, while 6.8 falls within correctable range.

The blind paired user study makes the same split explicit. Among archival photo scientists and historians, diffusion was called "more natural looking" in 71.5% of 400 pairs, but when asked "is this the original palette?" those same experts chose diffusion in only 41.8% and traditional in 58.2% (Hist10K User Study). The experts were answering two different questions, and the answers legitimately diverge.

On the 5,000 native black-and-white plates with no color ground truth, the three-historian panel flagged a substantial share of automatic diffusion outputs for palette anachronism — for example, neon-blue sky on an albumen print — versus a far smaller share for traditional scribble output (Hist10K Expert Review Protocol). The gap tracks the mechanism: modern neural colorizers are trained on paired color/desaturated images to learn statistically probable colors, as OldtoLife's 2026 guide summarizes, and those probabilities come from the modern world, not from historical emulsion chemistry. A scribble constraint forces the solver to respect the plate's existing brightness, texture, and shape anchors; an unconstrained diffusion pass can overrule them with a probable modern hue. This is why the Awesome-Image-Colorization outline separates user-guided scribble colorization from automatic methods: the constraint structure is not an add-on but the difference between anchored reconstruction and probabilistic invention.

Hist10K's dual-split design exists because the field lacked an established evaluation protocol; the ImageNet colorization benchmark was itself described as new to that role (Learning Representations for Automatic Colorization). A single FID score would have declared diffusion the winner. The two-sided scorecard is what exposes the myth that automatic diffusion is archive-ready because it looks natural: the same experts who preferred diffusion's naturalness in 71.5% of pairs judged it to be the original palette in only 41.8%, and the historian panel flagged a substantial share of automatic outputs for anachronism on genuinely historical plates. Naturalness is the wrong acceptance test for historical color.

| Hist10K metric | Diffusion | Traditional scribble | Which wins and why |
| --- | --- | --- | --- |
| FID, synthetic half | 8.9 | 15.3 | Diffusion — distribution plausibility |
| LPIPS, synthetic half | 0.092 | 0.127 | Diffusion — perceptual similarity |
| PSNR, synthetic half | 18.4 dB | 23.7 dB | Traditional — +5.3 dB reconstruction |
| CIEDE2000 ΔE | 12.1 | 6.8 | Traditional — nearly half the color error |
| "More natural" vote | 71.5% | 28.5% | Diffusion — screen plausibility |
| "Original palette" vote | 41.8% | 58.2% | Traditional — historical fidelity |
| Anachronism flag rate | High | Low | Traditional — large safety gap |

The decision rule falls out of the numbers: run the diffusion proposal inside a scribble-constraint and provenance-record loop. Where ground truth exists, the scorecard sets a concrete threshold — an archival output should meet the traditional side's 23.7 dB PSNR and 6.8 ΔE reconstruction floor, or the provenance record must document why it cannot. Without those two guards, the best diffusion output on this scorecard is a draft: visually natural, historically unreliable, and not a restoration.

![The Hist10K Scorecard — Hist10K: Evaluating Diffusion vs Traditional Colorization](https://static.mm-ais.com/article-images-pixabay/hist10k-evaluating-diffusion-vs-traditio-8d87c58a.jpg)

## The Hybrid Rule

The 90-day grant test settles the workflow before any aesthetic argument begins. At 10,000 images, pure traditional scribble colorization is labor-intensive enough that a 90-day digitization grant is exhausted before the batch clears review. The hybrid workflow's 2.1 minutes of expert review per image fits inside the same grant. Automatic diffusion looks faster only until the archive adds the 6.0 minutes per image of post-hoc verification required to catch hallucinated chroma; that variant does not fit either. The decision is a calendar decision, not a plausibility vote.

The threshold rule follows directly. Choose the hybrid workflow whenever the project batch exceeds 50 images. Choose pure traditional scribble colorization only for hero-image projects with fewer than 50 images and at least 10 minutes of human time budgeted per image. Choose automatic diffusion only for internal drafts, never for a catalog record—"looks natural on screen" is exactly the wrong test for historical color. ImageRestoreAI's 2025 pricing guide explains why the traditional route survives at small scale: professional human colorization is priced at $50-$300 per photo and historical specialist services at $200-$800+ per photo, which is defensible for a hero set and impossible for 10,000 records.

The hybrid workflow is a loop, not a magic setting. Run the diffusion proposal, add 40 auto-scribbles derived from saliency and luminance edges as hard chroma constraints, re-diffuse with those constraints, then compute a per-record decision flag. Per-image GPU time stays under 15 seconds on one A100, so diffusion's throughput advantage survives the constraint pass. The 40 scribbles are what preserve the traditional method's reconstruction-fidelity advantage; the decision flag is the provenance record. This aligns with Sik-Ho Tsang's scaling review, which tracks colorization models scaling to 100 million pretraining images—throughput scales, but constraints and provenance do not add themselves. The Awesome-Image-Colorization GitHub collection shows both automatic and user-guided diffusion tooling is mature enough to compose: the hybrid is an assembly of two existing toolchains, not new research infrastructure.

The winner row is explicit: hybrid is the only workflow that lets an archive say, per record, "this color is constrained by scribbles and proposed by diffusion." That sentence preserves enough of the traditional fidelity advantage while keeping diffusion's throughput advantage. It is also the only workflow that lands in the top half on all five rows of the comparison below.

| Metric | Automatic diffusion | Traditional scribble colorization | Hybrid |
| --- | --- | --- | --- |
| Throughput |

Canonical: https://colorizethis.io/blog/hist10k-evaluating-diffusion-vs-traditional-colorization.php
Markdown: https://colorizethis.io/blog/hist10k-evaluating-diffusion-vs-traditional-colorization.php/index.md