Understanding Image Noise in Your Film and Video Projects
Nothing ruins a shot like noise. Knowing what causes it and how to recognize it can save you a lot of frustration — and some useless footage.
There are few things as frustrating as noticing noise in an image while you’re shooting but being unable to identify the cause of the problem. In this article, we will take an in-depth look at noise by first reviewing some basic sensor science before examining noise in its various forms. So where to start?
Noise comes from a couple of sources. Reflected light streams into the lens and falls on the sensor, which is covered in millions of photosites — or pixels — that convert the variable levels of the light waves into digital signals. These small variances in current are what create images in modern sensors.
In the most common sensor for film production these days, the CMOS, attached to each photosite is an amplifier that adjusts the output of each pixel by cutting or boosting voltage, making the image darker or brighter, respectively. The user can do this by adjusting the gain or ISO. The data from the sensor gets read and the pixel charge resets.
Once the sensor information gets read, the data passes through a 12-bit analog-to-digital converter (frequently shortened to ADC or A/D) where the variance in voltage to each pixel gets converted to a binary value. Additionally, pixel location and other user-defined camera settings get saved as metadata in the camera storage device.
The majority of noise comes from the sensor or analog-to-digital conversion.
What Is Image Noise?
Put simply, image noise is undesired fluctuations of color or luminance that obscure detail in the shot you were trying to capture. (You can see examples of various kinds of noise over at Neat Video.)
Image noise arises primarily in underexposed footage as pixels have little light fluctuation to report in the intended image but are being over-amplified by boosted ISO values. Beyond exposure, sensors are also susceptible to a range of other issues that create noise in the final image. The heat of the sensor or other outside interference can also cause noise.
We can divide image noise into two categories: internal and interference.
Image noise originating from within the camera has a few root causes. The three main causes are electricity, heat, and sensor illumination levels.
In low-light situations where the sensor is being over-volted (ISO being pushed), each pixel has very little light wave fluctuation to report before being amplified. When you see noise in these situations, you are actually seeing the affected pixels reporting the fluctuations of the voltage of the pixel’s amplifier over the sensor.
Factors beyond the camera may also affect noise levels in the final image. This kind of noise is a bit rarer than internal noise in many modern cameras, but it can still affect the final image.
Interference noise is typically fairly easy to spot because it looks more like a pattern overlay on your image rather than the fluttering inherent to electronic noise. Mundane factors like strong radio transmissions in the area can also cause electronic noise — and they’re as hard to control as cosmic radiation.
What Are the Most Common Types of Video Noise?
- Gaussian Noise
Gaussian noise is a type of sensor noise. It’s primarily a side-effect of sensor heat. The heat is usually a result of voltage and illumination levels of the sensor.
Gaussian noise is most noticeable as a constant, fluttering across an underexposed, over-volted (ISO pushed to get exposure) frame. In most sensors, this noise will tend to be blue in underexposed images.
- Fixed-Pattern Noise
Fixed-pattern noise usually results from flaws in the manufacturing process of a particular sensor. It arises when different pixels have different levels of photosensitivity.
In video, this noise is relatively easy to spot due to the small variance in the output levels of affected pixels. FPN doesn’t move or chatter in the way other forms of electronic noise do. Instead, it appears as an overlay of pixels brighter than the actual information being recorded. It is easiest to spot in low-light situations, but other factors can exacerbate it as well.
- Salt and Pepper Noise
Technically referred to as either “Fat-tail distributed” or “Impulsive” noise, Salt and Pepper noise manifests as pixels erroneously reporting bright readouts in dark parts of the frame or dark readouts in bright parts. It looks similar to dead pixels, except Salt and Pepper noise will produce this effect randomly. Usually, analog-to-digital conversion or other errors in pixel interpretation cause this kind of noise.
Shot Noise is the main type of noise in darker parts of the image. Technically dubbed “Photon Shot Noise,” this type of noise results from the natural, inherent variation in photons striking each pixel at any given time, based on the exposure level. More technically, “statistical quantum fluctuations” cause this, but you get the idea.
Shot noise is the biggest cause of the “blockiness” in the shadows of an underexposed or over-volted shot. At extreme levels, Shot noise becomes Salt and Pepper noise.
Quantization is a term non-specific to image noise. Essentially, quantization reduces a large set of (usually) continuously changing values in order to get a workable total sum value or smaller representative set of output data. Quantizers have a set number of possible output values, and as the data is processed, it’s effectively rounded to one of these values.
Quantization is pretty simple to understand in video applications — millions of pixels’ individual readouts are quantized into a smaller representative set. In video circles, this is commonly referred to as “pixel binning.”
In video, Quantization noise is usually undetectable as all pixels will be more or less affected equally. Errors can occur when the image clips far beyond the extreme ends of quantization values of the A/D converter. It can also be made significantly worse by strong noise of other types in the image.
Anisotropic (An-Isotropic) noise arises when the sensor readout is sampled or quantized. This type of noise reduces perceived image resolution in affected shots by blending fine details together, creating patterns that aren’t actually there, or interpreting straight lines as jagged.
Anyone familiar with video should get this one pretty quickly — just think of the terrible aliasing and moire of the first several generations of DSLRs.
This type of noise in video cameras usually manifests when the native sensor resolution is much higher than that being recorded. Many older cameras opted to sample the higher resolution down to recorded resolution rather than simply scaling output to fit, causing much of the terrible aliasing and moire of early large-format video sensors.
Periodic noise is interference noise. It occurs when any number of natural or man-made signals interferes with the recorded signal. It typically appears as a fixed pattern overlay on top of the desired image.
When working with video, it is essential to learn to spot the kind of noise that is affecting your image as you shoot. For many types of image noise, there are fairly simple ways to reduce or negate the negative impacts on the footage if you know how to spot the correct type as you shoot. If you are seeing noise in your image, remember to step back from the camera, make note of your environment, and then run through a list of each type of noise in your head to develop a strategy to reduce your headache in post. Technical remedies will serve you well, but never discount a little bit of creative ingenuity.
Cover image via 25krunya.
Looking for more video production insight? Check out these articles.
- The Benefits of Working with 4K Footage in a 1080 Sequence
- Tips for Shooting Super High-Speed Footage with the Phantom Flex 4K
- Why Dual Native ISO Should Be the New Industry Standard
- What Else Can You Do with Your 360° and VR Video Footage?
- Revive Your Footage With Resolve 15’s Automatic Dirt Repair and Dust Buster Tools