Restoration & Noise Models
Filtering (Chapter 3) reshapes an image on purpose. Restoration tries to undo something that happened to it by accident.
5.1 The degradation model
Restoration assumes: "a clean picture existed, then something blurred it and/or added random static, and now I only see the ruined version." The whole job is working backward from the ruined version to guess the clean one — which is always a bit of a guess, never a perfect undo.
Image restoration usually starts from an explicit model of how a clean image got corrupted:
Think of a voice memo recorded in a echoey room with a hissing air conditioner in the background.
The echo is like h — it smears each word into the next, a kind of "blur" in time instead
of space. The AC hiss is like η — random noise added on top, unrelated to what's being
said. If you only have the recording, you can't perfectly recover the original clean voice — you can
only make an educated, statistically-informed guess, trading off "remove the echo" against "don't
amplify the hiss into a screech." Image restoration is the same negotiation, just with light instead
of sound.
f is the true underlying scene, h is some degradation function (motion blur,
defocus, sensor point-spread) applied by convolution, and η is additive noise. Restoration
is the attempt to invert this — recover f given only g, and possibly some
knowledge or estimate of h and the statistics of η. It's inherently an
ill-posed problem: noise makes exact inversion unstable, so every restoration method is really a
negotiation between removing degradation and not amplifying noise in the process.
5.2 Common noise models
"Noise" isn't one thing — random speckles, salt-and-pepper dots, and grainy texture are all different statistical patterns that call for different cleanup tools. Using the wrong tool for the noise you actually have is like using a lint roller to clean up spilled water.
Different sensors and situations tend to produce noise with different statistical character. Recognizing which one you're looking at determines which filter is appropriate — a filter matched to the wrong noise model will underperform even if it "sort of" removes noise.
Gaussian: the fine random grain you see in a photo taken in a dim room with high ISO — every pixel is nudged a little, randomly, independent of its neighbors. Salt & pepper: the scattered white/black dead pixels sometimes visible on an old or damaged scanner/camera sensor — isolated, extreme, all-or-nothing errors. Speckle: the grainy sparkle texture you see in an ultrasound scan, or in old laser-pointer dots on a wall — a signature of coherent light/sound interfering with itself. Poisson: why nighttime phone photos look grainiest in the darkest shadow areas, not the bright ones — counting very few photons is inherently less statistically reliable than counting many.
| Noise type | Character | Typical source | Matched filter |
|---|---|---|---|
| Gaussian | Random value added at every pixel, bell-curve distributed | Sensor/thermal/electronic noise | Mean/Gaussian smoothing, Wiener filter |
| Salt & pepper (impulse) | A fraction of pixels randomly forced to pure black or white | Bit errors, dead sensor elements, transmission faults | Median filter (nonlinear) |
| Speckle | Multiplicative — noise scales with signal, grainy texture | Coherent imaging: SAR, ultrasound, and often LiDAR intensity returns | Lee filter, Frost filter, or log-transform then Gaussian model |
| Poisson (shot noise) | Variance grows with signal brightness itself — noisier in bright regions | Photon-counting / low-light imaging | Variance-stabilizing transform (Anscombe) then Gaussian filtering |
Coherent-illumination sensors — SAR and, to a meaningful degree, LiDAR intensity returns — exhibit speckle rather than clean Gaussian noise, because the return signal is built from coherent interference of many sub-resolution scatterers. Applying a Gaussian-optimized filter (or, worse, a Gaussian noise assumption baked into a learned feature descriptor) to speckled LiDAR intensity data is a mismatch that a median or Lee filter — or at minimum a log transform before smoothing — usually handles better. This is worth testing explicitly rather than assumed away when picking preprocessing for a handcrafted-feature pipeline built on LiDAR intensity data.
5.3 Order-statistic (nonlinear) filters
Instead of averaging a neighborhood (which lets one extreme "dead pixel" drag the average off), sort the neighborhood's values and take the middle one. A single wild outlier can never win a "what's the middle value" contest, so it gets thrown out completely instead of blended in.
The median filter replaces each pixel with the median — not the mean — of its neighborhood. Because a median is far less swayed by a single extreme outlier than a mean is, this filter is dramatically better at removing salt-and-pepper noise while preserving edges, which a linear blur would smear.
If nine of your coworkers earn roughly $50k–70k and one outlier CEO earns $50 million, the average salary in that room is a wildly misleading number pulled way up by one extreme value. The median salary — the middle value if you lined everyone up in order — stays right around $55-60k, completely unmoved by the outlier. That's exactly why a median filter shrugs off a single corrupted "salt" or "pepper" pixel: sorting and taking the middle value is immune to one extreme number in a way that averaging never is.
5.4 The inverse and Wiener filters
If you know exactly how an image got blurred, you can in theory "divide out" the blur perfectly. In practice this amplifies noise catastrophically, so real methods deliberately hold back — accepting a little leftover blur in exchange for not turning every speck of noise into a scream.
When degradation is a known blur H(u,v) with no noise, restoration is in principle exact:
divide by H in the frequency domain.
Imagine trying to recover an original number by dividing by a near-zero blur value, say
0.001. Even a tiny bit of noise sitting on top — say a stray 0.01 — gets
divided by that same 0.001 too, turning a small, harmless wobble into a x10 amplified
spike in the result. Wherever the blur function H is weak, the noise doesn't just survive
the "undo" — it gets massively magnified, often to the point of drowning out the actual recovered
image entirely. The Wiener filter's whole trick is refusing to divide by numbers that are "too close
to zero to trust," settling for a softer, safer correction there instead.
In practice this blows up wherever H(u,v) is small or zero, because any noise present at
those frequencies gets divided by a near-zero number and amplified without bound. The Wiener
filter fixes this by incorporating the signal-to-noise ratio directly into the inversion,
trading a small amount of residual blur for a large reduction in noise amplification:
This is the general template behind most modern restoration: don't invert blindly, regularize the inversion using some estimate — explicit or learned — of how noisy the input actually is.
5.5 Why this matters before feature matching
Edge and corner detectors work by looking at how fast brightness changes — and noise is, by definition, fast, random change. Feed a detector noisy data and it will confidently report fake "corners" that are really just noise, not real terrain.
Every keypoint detector in Chapter 11 differentiates the image at least once, and differentiation amplifies high-frequency content — which is exactly where noise lives. Running a detector on a raw, unfiltered LiDAR intensity return will frequently produce spurious keypoints seeded by noise rather than by real terrain structure. A denoising pass matched to the actual noise character of the sensor — not a generic blur — is cheap insurance before any of the matching pipelines discussed later in this reference.