Chapter 1 - Image Filtering
Table of contents
Part 1 - Images and Noise
Slides 1-33
In the first part, we learn about digital images and the kind of degradations you get when capturing the real world with a digital camera, which suffers for example from finite pixel grid, finite range, and capturing noise. We will discuss different types of images and what you get when you vary dimension of domain and range. Finally, we give a classification and more formal treatment of image noise using notions from statistics.
Side note: unfortunately, the camera lost power around 10 minutes before the end of the recording, and the frame freezes. Apologies for this, but since it takes quite a bit of time to record and cut videos, I did not redo the recording. Same for small mistakes I make, which I correct using annotations whenever I find them.
Index
00:00 Overview
01:45 Digital images, sampling and aliasing
10:07 Quantization
13:10 Higher-dimensional domains and co-domains
20:05 Types of image noise, additive uniform noise
28:45 Additive Gaussian noise
35:04 Multiplicative noise
37:23 Salt-and-pepper noise
Notes and Errate
37:13 For multiplicative noise, you can actually see a distinctive characteristic as well, as the darker regions look visually less noisy than the brighter ones.
Part 2 - Correlation and Convolution
Slides 34-61
We show that to reduce image noise, we can average different samples of a pixel in the temporal domain. If we have only a single image, we can still try to locally average over a neighbourhood. This operation is the moving mean or moving average.
As a generalization, we introduce the correlation, where the entries of the filer are arbitrary. Since the correlation neither is commutative nor has an "identity image" which returns an unchanged copy of the filter when applying it, we modify the correlation slightly by flipping the filter before applying the correlation. The result is the convolution. We note important algebraic properties of the convolution, among the most interesting ones being linearity (or the superposition principle) and shift-invariance.
Finally, we consider other types of filters aside from blurring filters: sharpening, and convolutions which occur in optical systems.
Index
00:00:00 Overview and motivation
00:01:25 How to reduce noise? The idea of averaging
00:09:50 Local averaging: the moving mean, filter kernels
00:15:28 Moving average in 2D
00:20:10 Generalization to arbitrary filters: correlation
00:26:40 Boundary conditions
00:31:42 The Gaussian filter
00:41:14 Problems with the correlation from a mathematical perspective
00:46:46 The convolution
00:56:42 Shift-invariance
01:07:30 Sharpening filters
01:15:12 Convolution in optical systems
01:18:38 Summary and outlook
Notes and errata
00:03:00 In the previous part, I wrote $f$ for the observation and $g$ for the original, here it is accidentally the other way around.
00:20:10 I forgot to mention that a key generalization is now that the entries $h(k,l)$ of the filter can also be arbitrary now, so the filter does not have to compute an averaging operation. When sliding the filter over the image, the entry of the filter is multiplied with the respective pixel below it, as written in the formula.
00:41:14 Note: since this shows that $\delta \otimes h \neq h \otimes \delta$, the correlation is also not commutative, which I forgot to mention explicitly.
Part 3 - Non-linear filters
Slides 62-86
We note that while we motivated moving means and the convolution filter with an intention to denoise images, they actually do not perform very well for this use case due to abysmal performance at edges. We show some examples for simple, but efficient denoising filters which perform much better: the bilateral filter and non-local means, and discuss ways how to measure performance of image enhancement methods. We also think about problems when removing salt-and-pepper noise, and introduce the median filter which works particularly well for this use case (but also other types of noise).
Index
00:00 Introduction, measuring denoising performance and choice of parameters
11:25 Improving performance at edges - the bilateral filter
20:22 Discussion of denoising properties of the bilateral filter
23:46 Non-local means
34:49 Problems with applying these filters to salt and pepper noise
38:20 The median filter
44:42 Preview: Denoising by global optimization
49:34 Summary and outlook