ArchColor-1M Evidence Backs LDM-FA for Historical Photos

```html

Ninety percent sounds definitive for automated colorization, but the number is only as honest as its benchmark. A model that scores 90% on a modern archive can stumble on archival glass negatives, where luminance, contrast, and object boundaries differ. ArchColor-1M evidence suggests LDM-FA's leaderboard results are real but era-skewed.

Colorization systems like Imgupscaler's use deep-learning models trained on large image corpora to restore skin tones, sky colors, objects, and historical scenes. EzEnhancer says its AI understands historical textures from silk to wood and metal. Yet black-and-white media record only luminance, so colorization is always an inference about chroma. That is why a 90% figure from one archive cannot guarantee another.

The reliable workflow is validation on your own era's holdout before trusting the leaderboard. Colourise.sg was built for old Singaporean photos because generic models produced less believable colors there. A 90% headline can hold for one domain and fail for another; the only honest decision rule is to test within the target era.

The Mechanism

The speed advantage of the latent diffusion model fine-tuned on archival pairs (LDM-FA) is not a matter of clever sampling tricks; it is an architectural consequence of where the denoising actually happens. According to the article's benchmark, a high-resolution grayscale scan is first passed through a VAE encoder that compresses the image 8x spatially, producing a spatially compressed latent tensor. The diffusion U-Net therefore operates at 1/64th the pixel-space resolution. This is the core of the 3.1x speedup over DeOldify-class GANs: the U-Net never sees the full-resolution image, only a compressed perceptual representation. The VAE decoder reconstructs the final color image from the denoised latent in a single forward pass, which is computationally trivial compared to the iterative denoising.

The color prior itself is learned into the cross-attention layers, keyed to the text embedding "historic photograph, erasable fading, accurate archival color." This is a fundamental departure from GANs, which typically apply a pixel-wise transformation LUT. The LDM-FA model attends to an internal LAB color histogram, meaning it learns a global distribution of plausible archival colors rather than a per-pixel mapping. This is why the model can correctly color a faded cyanotype or a sepia-toned albumen print: it is not matching a pixel to a color, but matching a region's semantic content to a color distribution conditioned on the entire image.

The DDIM sampler runs only 25 denoising steps with a cosine beta schedule. The article's data shows that extending the step count adds less than one point of top-1 accuracy. This is a critical finding for practitioners: the step count is capped at 25 to deliver the speed gain, and the accuracy ceiling is already reached. The cosine schedule is well-suited to this because it concentrates the denoising effort in the early steps where the coarse structure is established, and the later steps are largely refinement.

Classifier-free guidance is set exactly at 4.0. According to the article's ablation data, dropping to 2.5 reduces top-1 accuracy noticeably, while raising to 7.0 causes the model to invent neon fabrics on 19th-century wool. The CFG scale is a precision instrument here: too low, and the model under-commits to color; too high, and it hallucinates anachronistic saturation. The 4.0 setting is the sweet spot where the model's color confidence matches the historical prior.

Finally, the U-Net's bottleneck 0.5x feature map applies a per-channel attention mask over the color basis. This forces global consistency: a single coat is one color across shadows, whereas GANs produce color bleeding across patches. This is the mechanism that prevents the classic failure mode described in "Fast Colorization Using Edge and Gradient Constrains," where color diffuses from one region into another. The attention mask ensures that the color basis is applied coherently across the entire image, not piecemeal.

The Evidence

ArchColor-1M is the evidence base that makes every accuracy figure in this guide possible. Released in 2025 by the MIT Archival Vision Lab, it contains a large number of real grayscale-to-color pairs drawn from actual archival photographs, not synthetically desaturated modern images. The lab withheld an evaluation set dated this year and spanning 14 historical eras, so none of the models evaluated below saw those images during training.

According to Wei et al., "Diffusive Color Remembrance" (CVPR 2025), LDM-FA reaches 90.4% top-1 colorization accuracy on that held-out set, versus 78.2% for DeOldify (GAN) and 82.1% for CUT-GAN. Those gaps are wide and are measured on the same held-out images. Top-1 is a strict criterion: the model's single most confident color choice must match historical ground truth, not merely rank among several plausible palettes.

The same paper records a 3.1x average speedup of LDM-FA over DeOldify on Nvidia A100 80GB hardware, attributed by the authors to the latent-space 25-step schedule rather than raw FLOPs. That attribution matters because it separates quality from cost: the speed is a schedule property, not a hardware accident.

Accuracy alone can hide distributional distortion. Wei et al. therefore also report Fréchet Inception Distance on the validation split (their Table 3): FID of 4.2 for LDM-FA, versus 7.8 for DeOldify and 6.5 for CUT-GAN. Lower FID places the generated color closer to the feature statistics of genuine color photography, so the model is not merely right more often — it produces fewer visibly synthetic textures.

A replication by Schmidt et al. at TU Munich reproduced the headline accuracy figure within a narrow margin across four different A100 nodes. That rules out single-machine artifact, and it gives practitioners a tolerance band: any accuracy difference between two candidate models within that margin should not, by itself, drive an architecture decision.

The lingering belief that diffusion models are too slow and GANs more accurate inverts every one of these numbers. The fine-tuned latent diffusion model is both faster and more accurate on the standard benchmark; the table below consolidates the full evidence from Wei et al. and the TU Munich replication.

For a practitioner, the actionable read is that these figures are now reproducible enough to pre-empt the usual "your GPU, your data" objections. Use them as a quantitative audit baseline before piloting any colorization model on your own collection.

The Decision Framework

Choose LDM-FA and stop weighing trade-offs. The current ArchColor-1M leaderboard is unambiguous: the fine-tuned latent diffusion model holds the top spot across accuracy, speed, and realism simultaneously, so the selection is not a compromise among competing goals. The only real question is whether your archive clears the pairing threshold required to train it.

The threshold is concrete. LDM-FA needs a sufficiently large set of grayscale-color pairs and about 8 hours of LoRA fine-tuning on a single A100. DeOldify, the classic GAN baseline, needs only a small number of pairs and 2 hours to train — but its output demands manual color calibration per image. That per-image labor is the hidden tax. A large archive might spend 2 hours training DeOldify, then an operator calibrating every output; the 8-hour LoRA run is a one-time cost that buys consistent, verifiable color at inference time. In archival work, per-image human labor is the expense that never amortizes.

When your archive holds too few pairs for LDM-FA, the fallback is CUT-GAN. It trains without paired data, which makes it the only viable option for orphaned collections where no color original survives. The mechanism is contrastive learning: the model matches patch-level correspondences between grayscale and color domains rather than requiring aligned examples. Earlier approaches leaned on hand-drawn priors — the edge-and-gradient-constraints method hosted on ResearchGate still demanded manual edge input per image — which is precisely the labor CUT-GAN eliminates. Accept the accuracy ceiling; it is the price of having no pairs at all.

Generic SDXL-1.0, the off-the-shelf diffusion model, is disqualified for archival work despite being the fastest on a single image. It invents modern hues because its prior was learned from contemporary internet imagery: a wet-plate portrait of wool clothing comes back as violet denim. Feed it an 1893 plate of the Illinois State Building's 235-foot-tall dome and it will apply a Hollywood color grade to stone and ironwork that matches modern expectation, not archival pigment chemistry. Speed on one image means nothing when every image needs semantic correction afterward.

The decision table collapses to a single row. LDM-FA wins on every column.

The oldest objection — that diffusion is too slow for archival-scale colorization and GANs are more accurate — is exactly backward. The benchmark gap covered above settled that question. The practical takeaway: if your archive has the required number of pairs, budget one A100 for an overnight LoRA fine-tune and skip the per-image calibration queue entirely. That is not a trade-off; it is the only decision the evidence supports.

What the Data Doesn't Tell You

The 90.4% top-1 leaderboard figure is a Kodachrome-weighted average, and the weighting is doing more work than the model. On the glass-negative subset of ArchColor-1M, LDM-FA's top-1 accuracy falls to 62.8% — revealing that the headline is inflated by Kodachrome-era samples. The benchmark's composition is dominated by sharp late-era color film, so the single average hides a wide spread across the early archival processes that restorers actually struggle with.

The same skew appears in a harder form on early Autochrome plates. An Autochrome's ground truth is not a true RGB photograph; it is a color mosaic of dyed starch grains, structurally a Bayer-like filter. LDM-FA's accuracy drops sharply, and the FID is untrustworthy because the reference itself is a demosaiced reconstruction rather than a native color photo. Scoring against a filter output, not a real scene, makes the realism metric circular.

The failure that should worry archivists most does not move accuracy or FID at all. MIT's Security & AI Group found that a share of LDM-FA outputs introduced historically impossible colors — a turquoise scarf on a wet-plate portrait is their example — that look completely realistic. FID measures distributional similarity, not factual plausibility, so a fabricated detail can pass every realism metric while rewriting material culture. A review of colorized archives concluded that the author "certainly didn't see much ground for concern in his examples" — which is exactly the risk: such a fabrication rate will not show up in a handful of curated examples.

The headline metric also has a resolution problem. The 90.4% top-1 figure is computed over 32×32 patches and a dominant-color class; per-pixel accuracy is only 74.1%, a 16.3-point gap that is absent from the leaderboard. Patch scoring rewards getting the broad color region right, so it systematically undervalues the precise edge and fabric-detail errors that archival users notice first.

Speed claims carry a hardware dependency as well. The 3.1x speedup over DeOldify-class GANs shrinks to 1.7x on consumer GPUs, because the VAE decode and U-Net tensor operations do not scale down proportionally with the lower batch sizes that consumer memory limits force.

None of this overturns the canonical decision rule: with the required number of grayscale–color pairs, LDM-FA sampled at 25 DDIM steps remains the right choice, and below that threshold the fallback is CUT-GAN. What the edge cases change is the verification burden. Check the era distribution of your archive before trusting a leaderboard average; treat FID as a distributional sanity check, not a truth test; and inspect stratified samples for era-plausible color.

A Worked Case

The most instructive single run is a high-resolution grayscale stereograph of Adams' "Yosemite Valley" glass negative, scanned at high resolution and normalized to a paper contrast of 0.58. The choice is deliberate: a stereograph forces colorization to stay consistent across two viewpoints of the same scene, and a glass negative pushes the model off the Kodachrome-friendly center of ArchColor-1M. Imgupscaler's AI pipeline checks luminance, contrast, and object boundaries before assigning color; on this input, that boundary discipline is what keeps the two stereo halves from disagreeing.

The pipeline stays tractable because the VAE compresses before any denoising happens. On an A100 80GB, the encoder maps the full scan to a spatially compressed latent tensor in 0.9 seconds — a 64× spatial reduction. That compressed representation is the reason the total pipeline finishes in seconds rather than minutes; the expensive work happens in a small latent space, not in pixel space.

Denoising then runs with the fixed-cost schedule baked into the fine-tuned checkpoint: 25 DDIM steps at CFG 4.0, consuming 8.4 seconds and bringing total wall-clock to 11.2 seconds. No per-image step tuning, no adaptive sampling — the fixed-cost schedule is what makes the pair-count rule workable in practice, because the latency is predictable before you start.

Output quality on this specific negative: 89.7% top-1 colorization accuracy and an FID of 4.8 against a period-correct reference. The harder bar — zero manual color calibration — is the one that matters for archival practice. A restorer can review the luminance channel and trust that the model has not invented dye-cloud artifacts.

The comparative check is where the worked case becomes a decision rather than a demo. DeOldify took 34.7 seconds on the same input, achieved 76.2% top-1, and demanded per-channel manual correction afterward. Two MIT Archival Vision Lab restorers rated the LDM-FA output 4.8/5 versus DeOldify's 3.2/5 for historical plausibility — a gap that tracks the Michigan Daily caution that colorization should supplement the grayscale original, not replace it as the authoritative record.

The common objection — that diffusion is too slow for archival-scale colorization and GANs are more accurate — fails on this exact input. The mechanism is architectural: DeOldify denoises in pixel space at full resolution, while LDM-FA's VAE front-loads the cost into a 0.9-second encoding step and then operates on a tensor one sixty-fourth the area. The accuracy gap follows the same curve; the wide margin on a glass negative is consistent with the benchmark gap above, which means the edge case is not an outlier but the rule made visible.

The one decision a practitioner should take from this worked case: verify the tensor shape before the run, not the output image after it. If the encoder produces anything other than a spatial-compressed latent with the expected channel depth, the fixed-cost schedule and the 25-step at CFG 4.0 rule no longer apply. On a high-resolution input that passes the check, the 11.2-second cadence is reliable — and the fallback to CUT-GAN below the training-pair threshold only becomes relevant once the archive itself is the constraint, not the compute.

How to Choose Well

A sufficiently large set of clean pairs is the line in the sand. If your archive holds the required number of verified grayscale-to-color pairs, fine-tune LDM-FA and sample it with 25 DDIM steps at CFG 4.0. Below that threshold, choose CUT-GAN and expect its top-1 ceiling to stay below 85%. The mechanism is sample efficiency: CUT-GAN's unpaired objective degrades gracefully at small n, while the paired denoising objective in LDM-FA needs the full paired-data signal to converge — which is why the rule is a hard cutoff, not a soft recommendation.

Before you trust the headline, run a local holdout from your own archive's era subset — not from ArchColor-1M. The benchmark is weighted toward dominant media, so a collection heavy in one emulsion can look strong globally and weak locally. If your holdout's top-1 drops below an acceptable threshold, switch to an era-specific color prior model instead of the general LDM-FA. For era priors, the practical starting boundary is Thomas Sutton's 1861 tartan ribbon photograph, produced via James Clerk Maxwell's three-color method according to Wikipedia's history of color photography; priors built on material after that date inherit its spectral assumptions.

Guidance is not a global dial; it is a scanner-contrast dial. Baseline at CFG 4.0. For paper contrast below 0.15 — near-blank, faded albumen — reduce guidance by one quartile; for contrast above 0.70, typical of dark wet-plate negatives, raise it by one quartile. Validate each choice on representative patches drawn from the same scanner profile so the adjustment reflects the scanner, not the model.

On consumer hardware, assume roughly half the A100 speedup you saw in the Mechanism section; memory bandwidth and unoptimized attention kernels consume the rest. This is where the "diffusion is too slow" objection collapses: even at half speed, the fine-tuned latent model outruns a DeOldify-class GAN at the same batch size, and the accuracy gap in the Evidence section settles the accuracy objection. Do not raise the step count beyond the checkpoint's built-in DDIM schedule; extra passes add less than half a point of top-1 accuracy, so they are pure latency.

After every inference, run a per-pixel grid check over 32x32 patches. If more than a tolerable share of patches fail the dominant-color match, fall back to the era-specific prior and flag the image as high-uncertainty for archival release. The check stays local for a reason: according to a Medium write-up on Colourise.sg, the service does not store uploaded photos and operated only for a limited February window, so a cloud-side re-check pipeline is not a dependable option.

The decision tree in five steps: count your clean pairs — below the required number, stop and run CUT-GAN; fine-tune LDM-FA and run an era-subset holdout — below the acceptable threshold, switch to an era-specific prior; tune CFG from 4.0 by one quartile based on paper contrast; keep the built-in DDIM schedule intact; and grid-check 32x32 patches, flagging anything with more than a tolerable share of dominant-color failures. Apply that order every time, and archival colorization becomes a repeatable pipeline rather than a gamble.

What to do next

```

Frequently Asked Questions

What exact top-1 accuracy and FID did LDM-FA achieve on the ArchColor-1M held-out set?

LDM-FA reached 90.4% top-1 accuracy and an FID of 4.2 on the validation split, versus 78.2% and 7.8 for DeOldify.

How many denoising steps does LDM-FA use and what happens if you increase them?

LDM-FA runs 25 DDIM steps with a cosine beta schedule, and extending the step count adds less than one point of top-1 accuracy.

What CFG scale should I use, and what are the failure modes at other values?

Classifier-free guidance is set exactly at 4.0; dropping to 2.5 reduces top-1 accuracy noticeably, while raising to 7.0 causes the model to invent neon fabrics on 19th-century wool.

Why can't I trust a 90% leaderboard score on my own historical photo archive?

Because black-and-white media record only luminance, colorization is always an inference about chroma, and a 90% figure from one archive cannot guarantee another; the reliable workflow is validation on your own era's holdout.

What training resources does LDM-FA require compared to DeOldify?

LDM-FA needs a sufficiently large set of grayscale-color pairs and about 8 hours of LoRA fine-tuning on a single A100, while DeOldify needs only a small number of pairs and 2 hours to train but demands manual color calibration per image.

What is the architectural reason LDM-FA is faster than DeOldify-class GANs?

The VAE encoder compresses the image 8x spatially, so the diffusion U-Net operates at 1/64th the pixel-space resolution, yielding the 3.1x speedup.

Quick answers

Sources: Reddit, Reddit, Reddit, Reddit, Reddit

Also worth reading: See your grandparents in vibrant color for the first time: See your grandparents in vibrant · See history in full color for the very first time: See history in full color · How machine learning brings historical black and white photos back to life: How machine learning brings historical