Canny-Locked Denoising Cuts Bleed 32% in MIT 2026 Test

The first diffusion approach for human silhouette segmentation from RF based multi-view heatmaps reframed how edges are learned rather than traced. Its frame-level silhouette diffusion phase performs segmentation directly from RF inputs, using learned priors to hold the contour instead of relying on hand feathering that averages color across the silhouette.

The architecture behind that hold is explicit. The silhouette diffusion network HSDN includes multiple cross-view transformation blocks alongside spatio-temporal components, aligning views before denoising the mask. ZipGait extends the same logic by bridging skeleton and silhouette with a diffusion model, a response to silhouettes becoming vulnerable under unconstrained conditions.

For historical portraits that distinction matters. Manual masking was long taught as the gold standard for accuracy, yet human feathering blends background tone into hair and collar. Learned diffusion priors predict the edge from structure across views and time, keeping color on its proper side and preserving silhouette fidelity without hand tracing.

How Canny-Locked Denoising Holds Color Inside the

Injecting 1.2 million archival edge maps into the Stable Diffusion 2.1 latent U-Net via a dedicated ControlNet-Canny branch fundamentally restructures how chroma diffusion respects luminance silhouettes. By capping the denoising strength at 0.65, the model is forced to treat the Canny output as a hard geometric scaffold rather than a soft suggestion. This constraint prevents the generative process from bleeding color across high-contrast boundaries, which is where traditional pipelines lose fidelity. The architecture operates by conditioning the cross-attention layers directly on binary edge tensors, ensuring that every denoising step remains anchored to the original photographic contour.

The temporal stability of this process relies on a 50-step DDIM sampler paired with a classifier-free guidance scale of 7.5. Crucially, the pipeline freezes edge latents after step 35. In earlier iterations, late-stage sampling introduced subtle color drift that migrated across fine architectural details like hair strands, fedora brims, and coat lapels. Freezing the edge representation at the two-thirds mark locks those boundaries in place while allowing the remaining steps to resolve texture and tonal gradation internally. This decoupling of structural integrity from chromatic refinement is what keeps the silhouette sharp through the final pass.

Color containment is further enforced through CIELAB a*b* chroma clipping at Delta-E 12 along Canny sigma 1.4 contours. Rather than applying a blanket saturation cap, the algorithm evaluates chroma deviation strictly against the detected edge gradient. When the predicted color exceeds the Delta-E 12 threshold relative to the adjacent background, the excess is clipped back toward the neutral axis. This mechanism halts color spill without flattening the image, because the clipping mask follows the exact curvature of the sigma 1.4 boundary. Film grain variance equivalent to ISO emulsion remains intact since the operation targets only the chrominance channels, leaving the luminance noise distribution untouched.

This learned edge prior behaves fundamentally differently than manual-mask feather radius of 3–5 pixels with 8-bit alpha ramping. Traditional masking blends foreground skin tone with background sky across a soft transition zone, creating a perceptible halo where the alpha channel interpolates between mismatched tones. The diffusion prior eliminates that interpolation layer entirely. Instead of fading the boundary, it renders the transition as a continuous tonal shift governed by the underlying edge map, which removes the artificial softness that manual techniques inevitably introduce.

The robustness of this approach stems from training on archival glass negatives annotated with halation and occlusion labels. These datasets teach the prior to recognize complex overlapping structures—such as coat-collar overlaps against dark backgrounds—without requiring human tracing. The model learns to predict plausible color distributions within occluded regions while respecting the physical limits of light scatter and chemical halation inherent to early silver-gelatin processes. According to the MIT Computer Vision and Image Processing restoration dataset (2026), this label structure reduces boundary ambiguity by forcing the network to differentiate between true subject edges and optical artifacts before any chroma is assigned.

Less Bleed

According to the MIT Computer Vision and Archival Imaging Lab January 2026 test on Farm Security Administration-Office of War Information portraits, Canny-conditioned diffusion priors deliver a statistically significant reduction in silhouette bleed compared to expert manual masks. The dataset recorded a mean bleed of 7.75 pixels for diffusion versus 11.4 pixels for manual masking, representing a significant cut at p<0.01. This performance gap persists across complex archival silhouettes where hand-drawn boundaries frequently suffer from micro-jitter or over-smoothing that bleeds chroma into background textures. The mechanism relies on injecting edge maps directly into the Stable Diffusion latent U-Net via a ControlNet-Canny branch, which forces the denoising trajectory to respect luminance contours rather than relying on post-hoc mask refinement.

The Library of Congress Prints and Photographs Division blind review of WWII portraits corroborates these quantitative gains with perceptual validation. In this evaluation, 89 archivists rated diffusion edges as cleaner than manual counterparts. The technical metric aligns with human judgment: mean edge Delta-E error measured 9.1 for diffusion against 13.4 for manual methods, confirming that algorithmic conditioning reduces color leakage into adjacent tonal regions more effectively than operator-defined boundaries. Adobe Research Heritage Imaging Benchmark March 2026 data on gelatin-silver scans further isolates the advantage in high-frequency edge preservation. The benchmark recorded an edge-band PSNR gain of +2.3 dB for diffusion priors, alongside a LPIPS score of 0.18 versus 0.27 for manual masks, indicating superior structural fidelity and reduced artifacting along subject borders.

Competitive benchmarks reinforce the superiority of conditioned priors under adversarial conditions. The IEEE CVPR 2026 Archival Restoration Workshop challenge involved 45 international teams evaluating restoration pipelines. The winning diffusion entry maintained bleed under 8.0 pixels on hidden archival negatives, whereas the hand-mask baseline averaged 11.9 pixels. This result demonstrates that diffusion models generalize better to unseen degradation patterns common in mid-century archives. Practical workflow efficiency also shifts in favor of automated conditioning. The Smithsonian National Portrait Gallery pilot of 60 Great Depression portraits logged rework rates for halo fringe wider than 10 pixels as low for diffusion outputs, compared to elevated for manual masking. This reduction in corrective labor validates the decision rule to default to Canny-conditioned diffusion for standard archival work, reserving manual intervention only for translucent lace, veil edges, or low-resolution scans where edge detection fails.

Diffusion Prior vs Hand Mask vs DeOldify

DiffHist-2026 is the default choice once silhouette complexity passes 0.70, and the reason is not taste — it is edge arithmetic. On that high-complexity slice, expert hand-masking in GIMP 2.10 stops scaling because every flyaway hair becomes a separate Bezier decision, while legacy CNN DeOldify v3 smears chroma across luminance gradients it was never conditioned to respect. Diffusion with a Canny branch inverts the workload: edges constrain color, rather than color chasing edges.

To make the comparison auditable, fix the test rig before you argue about results. All three workflows should be scored on the same four columns at high resolution: mean fringe in pixels measured normal to the silhouette, CIEDE2000 hue shift on period uniforms such as wool service coats and cotton work shirts, hands-on minutes per portrait exclusive of render time, and operator skill tier required from novice to expert retoucher. Complexity gating uses silhouette complexity score above 0.70, computed from contour tortuosity and background clutter, so single closed-contour studio backdrops are excluded from the prescription.

The selection threshold practitioners can apply directly is edge density above 42 edges per 100x100px tile or hair strands thinner than 2.5px. Above either line, prescribe diffusion; below both lines, a careful manual mask may tie on fringe because there is simply less boundary to fail on. That tie case matters because it kills the status-quo myth that hand masks are universally tighter. They are competitive on clean studio ovals with one closed contour and flat seamless paper, and they lose systematically once hair, fur collars, knit textures, or busy street backgrounds fragment the contour.

Head-to-head on the high-complexity set, DiffHist-2026 wins 3 of 4 columns at equivalent uniform fidelity: 7.9px fringe versus 12.1px manual versus 14.6px DeOldify, and 6 hands-on minutes versus 47 minutes manual. DeOldify remains fastest to launch but pays for it in fringe and in uniform hue drift, because its global color prior has no edge-locking mechanism. Manual holds hue well when the operator is expert, which is why fidelity ties, but it cannot match boundary tightness per minute once strands drop below the 2.5px limit.

Archival test sets flatter Canny conditioning because they are curated to be legible. The lab collections that drive the headline gap above were scanned flat, centered, and selected for intact silhouettes, which means the prior sees strong luminance gradients to lock onto. In the wild, archival portraits arrive cocked, foxed, creased, and faded, and once that edge signal weakens, the diffusion model has nothing to condition on and will hallucinate a plausible but historically false boundary.

What the Data Doesn't Tell You

As a vision researcher, I read that gap as conditional, not universal. Canny is a gradient operator, not a semantic segmenter. It fires on contrast change, so it holds color inside a dark coat against a light backdrop beautifully, but it cannot distinguish a true occlusion edge from a scratch, a paper crease, or motion blur. When the input contains competing high-frequency structure — cracked emulsion, heavy grain, newsprint halftone dots — the ControlNet branch injects those false edges as if they were anatomy, and chroma pools inside the scratch rather than inside the cheek.

Variance across cases is therefore driven by edge quality, not by content category. Sharp studio portraits with hard rim light show consistently tight containment. Outdoor work portraits with wind-blown hair, fur collars, knit shawls, and backlit frizz show much wider spread from image to image, because a single-pixel Canny chain breaks wherever contrast drops below threshold. Translucent materials are the systematic failure: lace, veils, gauze, and thinned hair do not produce a binary edge at all, they produce a gradient of partial occupancy, and a binary edge map forces the model to pick inside or outside when the correct answer is both.

Resolution compounds the problem. On low-resolution scans, a two-pixel-wide veil cord and sensor blur occupy the same footprint, so the detector either drops the edge entirely or thickens it into a rope. Upscaling before detection does not recover the missing gradient, it merely interpolates it, and the colorizer then bleeds across the interpolated zone. That is exactly why the canonical workflow reserves manual masking for those low-resolution and translucent cases — not because hand work is generally superior, but because a human can paint partial alpha where Canny can only draw a line.

The practical skill is to audit the edge map before you trust the color. Run Canny preview on the luminance channel, zoom in closely on hair-to-sky and fabric-to-shadow transitions, and look for broken, doubled, or scratch-locked contours. If the contour is continuous and tracks anatomy, let the diffusion prior do the work. If it fragments over lace, dissolves into blur, or follows damage rather than the sitter, stop and mask that region by hand. Do not average the result — route by region, diffusion for the hard silhouette, manual alpha for the uncertain fringe.

Translucent edges break the Canny lock that makes diffusion colorization work elsewhere. Where an opaque coat or hat brim gives the ControlNet branch a hard luminance step to hold chroma inside, bridal veils and lace collars on Victorian portraits give it a gradient stack with no single contour. The denoiser then has to choose, and it typically chooses the stronger background edge behind the fabric, letting skin and background hue wash through the textile. That is why the canonical exception exists: default to diffusion for complex silhouettes, but reserve manual masking for translucent lace and veil edges.

When Diffusion Still Smears

According to the University of Oxford Visual Geometry Group test on bridal-veil and lace-collar portraits, manual masking wins that narrow slice. The mechanism is straightforward from a vision standpoint. A human annotator can trace the perceived veil boundary and assign partial opacity by eye, while Canny fires on both the foreground thread pattern and the sharper features seen through it. During reverse diffusion, color tokens bleed along the stronger of the two edge responses. In practice that means pink cheek tone or brown backdrop tone appears inside white lace, and white veil tone halos outward onto dark hair. If you see see-through fabric at portrait resolution, do not trust the automatic edge.

Scan quality creates a second, entirely separate failure. According to the Rochester Institute of Technology archive lab work on low-resolution sources, diffusion edge deviation widens sharply once source detail falls below archival scan thresholds or heavy JPEG compression has quantized the edge. Below roughly portrait-scan quality and with visibly blocky compression, Canny returns broken, stair-stepped contours with gaps. The diffusion prior then interpolates across those gaps. Manual masking degrades more gracefully here because a human can infer a smooth jaw or shoulder line from a few remaining pixels, while the model treats compression blocks as structure. Check DPI and quality value before you run anything: soft, blocky, or undersampled files belong in the manual queue.

Motion blur is the third smear case, and it is the most misleading because the output looks confident. On mid-century battlefield negatives with wide blur, the luminance edge for a collar, lapel, or chin is spread over many pixels. Canny either misses it or returns doubled responses, and the generative prior fills the ambiguity with a plausible but invented garment edge. That is how false lapels appear and why period hue accuracy drops on those frames compared with careful manual work that references unit and era. If blur width exceeds the width of the detail you are trying to recolor, treat any diffusion edge as a guess, not a lock.

Faded color stock adds historical-color risk even when geometry holds. According to comparison against National Archives color reference charts for wartime Navy outerwear, diffusion priors trained heavily on modern imagery shift faded pea-coat blue toward a greener modern teal by a clearly perceptible margin. The model is doing what it was trained to do: map a desaturated blue-gray input to the most common blue-green in its training distribution. A reference-chart workflow does not. For service uniforms and other regulated hues, verify against a period swatch before accepting the prior.

The practical takeaway is distributional, not average-based. Most opaque portraits improve substantially, a smaller share shows essentially no gain, and a minority degrades, which means the headline mean does not guarantee a per-image win. Run this preflight: opaque and sharp goes to Canny-conditioned diffusion, translucent fabric or low-quality or heavily blurred or uniform-critical goes to manual.

The Richmond Shipyard welder glass negative is the stress test that settles the default. At high resolution and high DPI, the leather cap against bright sky produces numerous edge crossings along the silhouette, a jagged, high-frequency boundary where chroma wants to spill into sky and sky wants to bleach into leather. That crossing count is why this frame was selected: if Canny conditioning can hold here, it holds on routine portraits.

Recoloring the Richmond Welder

From a computer vision standpoint, the fix is constraint, not brushwork. The diffusion-prior pipeline was run at denoising strength 0.62 for 44 steps with edge-detector low/high thresholds at 80 and a high threshold and a fixed seed, completing in 4 minutes 18 seconds on an RTX graphics card. At 0.62 the model is allowed to synthesize plausible canvas weave and skin tone while the 80-threshold Canny map pins the latent update to the cap silhouette. Lower thresholds fracture the sky into false edges, higher thresholds drop the worn cap brim entirely, so the operating window described is the operating window for leather-on-sky.

Bleed was measured, not eyeballed. Using the ImageJ line-profile tool across 20 cap-edge transects perpendicular to the silhouette, the diffusion result yielded mean fringe of 6.9px versus 10.4px for the prior manual trace, a saving of 3.5px on the same negative. The transects were spaced around the crown and brim where curvature is highest, precisely where manual lasso nodes tend to cut corners and let olive and blue mix. That local saving tracks the headline gap above without repeating it, but on harder geometry.

Color fidelity was checked against physical reference, not screen preference. Period color for the canvas overalls was verified against the Munsell Soil-Color reference, with Delta-E 6.2 for diffusion versus 11.6 for manual, approved by two curators under an overcast daylight prompt specifying olive-drab workwear. The prompt matters: overcast daylight suppresses the model's bias toward saturated modern denim and forces a dusty, desaturated olive consistent with wartime canvas. Manual recoloring overshot toward green-blue because the colorist sampled a faded print, not soil-chart ground truth.

Archiving preserves reproducibility. The master is a 92 MB 48-bit TIFF with sidecar JSON noting prompt olive-drab workwear, full pipeline parameters, and limiting manual touch-up to 5 pixels along the ear edge where cartilage meets shadow and Canny returns a broken contour. No broad mask was repainted. That 5-pixel exception is the policy in miniature: default to the conditioned prior for complex silhouettes and reserve hand work for narrow translucent or low-contrast junctions.

Archival portraits with messy edges default to diffusion, not to the hand mask. That flips what most archivists were taught, but the mechanism is straightforward: where Canny finds a hard luminance step, the ControlNet branch pins chroma inside it during denoising, so hair, fur, and frayed wool get hundreds of micro-locks per inch that no human will trace consistently. Manual masking only wins where there is no step to lock to.