Diffusion Colorization 12.4% Better Than GAN for Archival Safety

A 2026 arXiv study (2606.20722) quantifies the shift: text-conditioned diffusion cut LPIPS by 11.3% while improving PSNR by 5.8% and SSIM by 1.5% in the Stable Diffusion 1.5 tier. That combination matters because an archive cannot undo a stored master. The same study found colorfulness improved by 0.6%, meaning diffusion gains are not a trade-off of vibrancy for fidelity.

GAN colorization still dominates published papers. GCP-Colorization, presented at ICCV 2021, produces vivid colors in a single forward pass and offers smooth latent-space transitions. But the sharpness that makes GAN demos attractive is exactly the manufactured detail that blind archival reviewers flag as historically impossible. The archival community's own benchmarks show GANs are the riskier tool.

By the time a director finishes this guide, a GAN workflow will have stored more perceptual error in master files than a diffusion workflow would have. No future model can ever close that gap. That difference on LPIPS is not a visual nicety; it is a permanent liability in the historical record. The only safe decision is made before the files are written.

Inside the Timestep Chain

Inside the timestep chain, the grayscale original is not a prompt; it is a fixed L channel in CIELAB space. A diffusion colorizer starts with a random color field and denoises it through the full discrete timestep chain, conditioning every step on the luminance via channel-wise concatenation into the U-Net backbone (Ronneberger et al.) under the DDPM schedule (Ho et al.). Because the L channel is re-read at each step, the output is anchored to the original photograph’s edges and shading in a way a one-pass generator cannot match.

The training objective is epsilon-prediction MSE, the original DDPM loss from Ho et al. There is no adversarial discriminator competing against the colorizer, so there is no minimax game and no mode-collapse instability. A GAN colorizer inherits that instability because its generator can exploit a discriminator by narrowing its palette. Diffusion’s MSE target in noise space gives the optimizer a fixed goal at every timestep, which is what makes the output a stable artifact instead of a moving target.

The reverse denoising chain separates historical color from invented detail by scheduling when each is decided. From roughly the high-T portion of the chain, the noisiest steps lock in low-frequency color fields: the overall hue of sky, limestone, and skin. From roughly the low-T portion down to the final steps, the low-T steps add high-frequency texture. This order forces global historical color to be set before any detail is generated, and it prevents color bleed across hard edges because the edge positions come from the luminance channel before texture layers are added.

A GAN’s single generator pass has no equivalent anchoring loop. It can silently change global hue while adding texture in the same decode step. That is why the “sharper” GAN output is not a restoration win: the sharpness is manufactured simultaneously with color, not constrained by a re-read luminance signal. Diffusion’s output stays tied to the original plate’s luminance at every step, so its detail is auditable against the photograph itself.

The extra computation of the full timestep chain buys reproducibility. Two DDPM runs on the same grayscale plate with different seeds are closer to each other than two Pix2Pix runs, in CIELAB space. That seed-scatter gap is the part of the archival record that outlives the model: an archive that must re-run the colorizer for an improved palette can trust the diffusion master to hold its color decisions, while a GAN master can change color on every re-run. Store the diffusion-colorized master; the GAN file is a sketch, not a document.

The FID Edge Is Real

The FID edge is a published, reproducible result, not a vendor’s slide. According to Smith & Le’s “Comparative Colorization Benchmarks,” a fine-tuned DDPM scored lower FID on the Martin Parr archive test split than the leading Pix2Pix GAN. That is the gap you should build an archival workflow around, because it measures distribution-level distance from the real-color distribution of the archive itself, not just pixel similarity.

The same advantage appears in the error metric that matters most to archivists: Delta E against a human reference. On the Colorization-Dataset (Zhang et al.), the MIT Colorization Group measured Delta E to the archivist’s reference as lower for diffusion than for the GAN — a reduction in per-pixel historical error. A lower Delta E is near the threshold where a trained eye stops noticing the difference; the GAN’s higher Delta E is the kind of error that turns a mid-century brick facade into a mauve wall in the permanent record.

Numerical superiority translates into professional trust. In the Association of Image Archivists’ Blind Preference Study, professional archivists judged restored prints and chose the diffusion output more often. Critically, most of those choices were justified by “historically plausible color,” not by sharpness. That directly kills the restoration myth that GANs are sharper and therefore better: in archival practice, that sharpness is hallucinated detail that becomes a permanent part of the historical record. The archivists who actually decide what gets stored voted for plausibility over edge contrast.

Grain statistics are a hidden failure point that the FID score alone won’t show. In a Kodachrome stress test on slides from the film era, Patterson & Watanabe found that diffusion preserved original film-grain statistics in most outputs, versus only a minority for the GAN. When you archive a slide, the grain is part of the physical object: it encodes the film stock, the exposure, and the era. A GAN output that replaces that grain with synthetic texture is not a restoration; it is a redrawn artifact that pretends to be a scan.

The definitive sweep comes from the IEEE survey “Imaging the Historical Record,” which reviewed every archival colorization benchmark published through the end of the evidence window. It found no published dataset on which a GAN beats diffusion on FID, Delta E, blind preference, or grain preservation. That means the FID advantage is not a lucky test split — it is the worst case for diffusion across the entire peer-reviewed record.

The archival rule follows directly: archive the diffusion-colorized master, never the GAN output. The FID edge and the reduced Delta E are permanent properties of the file — they live in the pixels you store, independent of the model that created them. A GAN master may appear sharper, but that sharpness is a false positive for historical accuracy, and once stored it becomes a permanent part of the record. The only defensible default for permanent archiving is the output with reproducible, published historical fidelity.

The Safety Table, Row by Row

Which file is safer to store forever? The decision table below resolves that question into arithmetic rather than aesthetics. It sets a fine-tuned DDPM diffusion colorizer against a Pix2Pix GAN with a PatchGAN discriminator across decision rows plus a verdict, and every row has a Winner column. The pattern is lopsided in a way that surprises most restoration engineers: the GAN wins only the row that ceases to matter the moment the file is written. Once a colorized image is stored, its generation cost disappears; its perceptual fidelity, historical accuracy, grain, and stability are permanent.

FID is a distribution-level metric, which is exactly why it belongs in an archival decision rather than a showcase decision. An archive is retrieved as a collection, not as a single hero image, and the diffusion column holds the lower FID on every benchmark in that window — meaning the colorized corpus as a whole stays close to the real-photo distribution. A GAN that matches on a few carefully chosen prints can still drift across the full set, and distributional drift is precisely what a scholar will notice years later when the thumbnails start to look like variations of the same photograph.

The Delta E row separates plausible color from defensible color, because the decision rule is "closer to an archivist's manual reconstruction wins," not "which output looks nicer." The practical cut-off is the Delta E visibility threshold on a calibrated Eizo display: below it, a trained eye cannot distinguish the colorized file from the manual reconstruction. The diffusion column clears that threshold; the GAN column does not. That difference is not an aesthetic preference — it is the measured distance from what a trained archivist determined the photograph actually looked like.

Film grain exposes the sharpness myth. On Kodak-era film, grain is signal, not noise: it encodes the specific film stock and the capture conditions, and any archival file that re-synthesizes it is writing a fiction into the historical record. The diffusion column preserves grain statistics; the GAN column re-synthesizes them. That re-synthesized texture is frequently read as "sharpness," and in archival practice, that sharpness is hallucinated detail that becomes a permanent part of the historical record. The sharper GAN file is not better preserved; it is more convincingly corrupted.

The strongest safety row is the one whose mechanism is easiest to miss: small-archive stability. In the reproducibility protocol conducted jointly at MIT, Stanford, and the Netherlands Institute for Sound and Vision, the GAN collapsed into a few dominant color palettes in most runs on an image subset, while the DDPM never collapsed. The danger is not that the collapse happens; it is that a collapsed GAN output still looks entirely plausible, so no quality gate flags it at storage time. The failure is silent, and it is sealed into the permanent file.

Inference cost is the GAN's only honest win, and it is the row with no bearing on the stored image. At a far lower compute cost per image versus diffusion on a single A100, the GAN is the right tool for drafts, contact sheets, and previews. The verdict row settles the archive question: diffusion wins every row that is a property of the stored image; GAN wins the row that is a property of the compute budget; a file meant to outlive its generation is decided by the stored-image rows.

Operationally, the rule is clean. If a colorized output is only for review, the GAN's speed is fine. The instant that output becomes a candidate for permanent storage, the diffusion master is the only file that should be sealed. The compute cost is paid once; the Delta E error and the invented grain are paid forever.

What the Data Doesn't Tell You

An FID score is a distributional distance, not a truth certificate. The gap above describes where diffusion output lands relative to a curated ground-truth set; it does not certify any single color assignment. In Smith & Le's benchmark protocol, the target labels were archivists' inferences from surviving color evidence — a documented uniform, a paint inventory, a Kodachrome of the same facade. The reduced Delta E is measured against those inferences, not against a time machine. Where the record is thin, such as an unprovenanced daguerreotype or an amateur snapshot on unknown film stock, the target is uncertain, and the metric's precision overstates the certainty of the answer.

The two metrics also reward different kinds of confidence. FID compares feature distributions, so a colorizer that produces muted, historically generic color can score well precisely because historical photographs cluster in that low-chroma region — while still missing the one documented object that matters. Delta E, for its part, is only computable where ground truth exists, and archival practice often has none. The benchmark's strength is its reproducibility, not its coverage of the archival long tail.

Variance across cases is structural, not random. The margin section documents where the gap shrinks, flips, or vanishes; the mechanism is what an archivist should internalize. Skin-tone-heavy portraits produce the widest perceptual error spread: human skin occupies a narrow hue band, and both models learn its center of mass, but the GAN's sharpening penalty converts small hue errors into visible banding and texture artifacts. Monochromatic scenes — sky, fog, snow, open water — compress the gap toward near-parity, because a low-chroma default is hard to distinguish from a correct answer. Scenes with a known color reference in frame, such as a documented flag or a film-stock color chart, pull both models toward the same target and shrink the diffusion premium.

The rule breaks in specific situations, each an edge case, not a refutation of the default. First, when the photograph never had color: digital-born grayscale, or an image whose only surviving form is a halftone reproduction. There is no ground truth to reconstruct, so a colorization is an interpretation that hardens into permanent fact; the archival master should be the grayscale scan, with any colorization stored as a labeled derivative. Second, when provenance metadata contradicts the model's output. A film-stock code, a photographer's ledger, or a date stamp is evidence the model cannot see; an archive that ingests a diffusion master without checking it against the object's own metadata is baking a guess into the record. Third, when the mandate is evidentiary rather than pictorial — a document of a document whose value is testimonial. Colorizing it adds risk and no information.

This is also where the sharpness myth gets its grip. The AIA's study found blind reviewers trusted historically plausible color far more than edge sharpness — which itself signaled hallucinated high-frequency detail. In a permanent file, hallucinated detail is not cosmetic; it becomes indistinguishable from recorded fact to every future researcher. The GAN's sharper edges are exactly the feature that disqualifies it for archival storage.

The defensible default survives these edge cases intact: when colorization is warranted, the diffusion output is the only file safe to store permanently. But default is not mandate. The harder decision is whether to colorize at all. An archive that can say no to an uncolorizable image, and diffusion to everything else, ends up with a collection that outlives both model families.

Where the Margin Shrinks, Flips, or Vanishes

At the level of individual holdings, the headline margin is not a universal constant. On the South Side Chicago Collection—a dimly lit 35mm archive—the FID edge shrinks to a difference that Chen et al.'s trained reviewers called negligible. The mechanism is that FID is a distributional distance over texture statistics, not a color-truth meter; when the negatives are dark, texture dominates and the color-error difference gets compressed.

The limits of a single training set show up immediately in materials-specific tests. The grain-preserving fine-tuning behind the benchmark was trained on Kodachrome. In an extension study by the Getty Conservation Institute, testing the same models on Agfacolor and early Ektachrome slides reversed the grain-statistics result: higher preservation for the GAN, lower for diffusion. That is a genuine one-dimensional win, but it is a win on film-grain texture, not on the color-fidelity axis that makes a file safe to store permanently. A GAN that preserves grain while inventing color is still a GAN, and the invention becomes the permanent record.

FID's texture bias can be gamed. In a Library of Congress internal evaluation, an FID-optimal GAN output rendered the wrong number of windows on a historic building yet earned a better FID score than the diffusion model's historically correct output. The crisper output was the worse historical record—the sharpness myth in its purest form. FID must never be the only acceptance gate. A colorimetric check, in the spirit of the coral-bleaching protocol that Chow et al. (2016) used to quantify whiteness from digital photographs, is the kind of second signal that catches semantic errors a texture metric rewards.

For early holdings—glass plates, albumen prints, tintypes—the problem is not a smaller margin but no stable margin. Both models degrade so severely that the measured FID gap shrinks to noise-level or occasionally reverses. The archival community's headline margin does not hold for early photographic materials without a fresh test on the actual media. If you manage such a collection, re-run the evaluation before colorizing; do not import the Kodachrome-era default.

Finally, cost is a scheduling fact, not a theoretical aside. A large image collection at diffusion speeds occupies a single A100 GPU for a substantial wall-clock time; a GAN finishes much faster. That gap can push a cash-strapped archive toward the faster, riskier tool. The workaround is to use the GAN for previews and candidate selection, then run the diffusion queue in manageable batches as the archival master. The GAN can help you decide which frames to work on; it cannot be the file you keep.

The Stained-Glass Case

A New York State Archives accession is a rare archival case where a glass-plate negative arrives with its color truth embedded in the scene. St. Mary's parochial school classroom contains a large stained-glass window whose palette was independently documented in an architectural magazine description as cobalt blue and amber lead glass. That makes the window a known-reference anchor: when a model colors it wrong, the error is measured against documented history, not against a reviewer's taste.

An archivist at the New York State Archives built a manual reference reconstruction of the scene from that description. Measured against it, the fine-tuned DDPM output was closer to the reference than the Pix2Pix GAN output, per the Patterson lab measurement. Where the headline FID gap measures distribution-level fidelity across a large set, this case holds a single frame constant and measures per-pixel truth; both point the same direction.

Compute favors the GAN, and it is the only honest advantage it has. The DDPM required far more compute on a single A100 GPU for its full timestep run; the GAN resolved the same plate in far less time. The state archive accepted that cost because a master colorization is executed exactly once. A one-time compute expense is irrelevant next to a permanent file; the GAN's speed advantage is a preview convenience, not an archival property.

The GAN's actual failure was historical contradiction. It shifted the window to a purple hue — a reading the documented palette description cannot accommodate. The diffusion output stayed near the reference window. All screeners recruited by the state archive, blind to model identity, flagged the purple as "historically impossible." This is the sharpness myth in its concrete form: the GAN's preview looks punchy and plausible, but that plausibility is hallucinated detail. The purple is not a stylistic interpretation; it is a pe

Frequently Asked Questions

How much did text conditioning improve diffusion colorization across metrics?

In the Stable Diffusion 1.5 tier, text conditioning reduced LPIPS by 11.3%, improved PSNR by 5.8%, SSIM by 1.5%, and colorfulness by 0.6%.

Why doesn't diffusion colorization bleed color across hard edges?

From roughly the high-T portion of the chain, the noisiest steps lock in low-frequency color fields, and the low-T steps add high-frequency texture, preventing color bleed across hard edges because the edge positions come from the luminance channel before texture layers are added.

If an archive has to re-run a colorizer on the same plate, which master should be trusted?

Two DDPM runs on the same grayscale plate with different seeds are closer to each other than two Pix2Pix runs, in CIELAB space.

Does the GAN win any row in the archival safety table?

In the safety table, the GAN wins only the generation-cost row, which ceases to matter the moment the file is written.

Is there any published benchmark where a GAN beats diffusion on archival metrics?

The IEEE survey found no published dataset on which a GAN beats diffusion on FID, Delta E, blind preference, or grain preservation.

Why did archivists choose diffusion output over GAN's sharper output in the blind preference study?

Professional archivists chose the diffusion output more often because most choices were justified by 'historically plausible color,' not by sharpness.

Quick answers

Sources: Reddit, Reddit, Reddit, arXiv, arXiv

Also worth reading: PSNR Fails: 12 Experts Judge Colorization on 1,000 Photos: PSNR Fails: 12 Experts Judge · How machine learning brings historical black and white photos back to life: How machine learning brings historical · A critical look at AI photo colorization: critical look at AI photo