PrimeEnhance

Author: Rachit Mohindra, Product Manager, Photometrics

2D Active Image Denoising

There are several sources of noise when imaging faint signal levels which can affect the Signal-to-Noise Ratio (SNR) of your measurement, the main types being dark noise, read noise and shot noise. Camera manufacturers make design choices to minimize the presence of noise in the image and to maximize the quality and SNR of the collected images. Dark Noise is reduced by cooling the sensor, and read noise is minimized through sensor performance and electronic design.

Photon shot noise however, is an inherent property of light. There is always a statistical variation in the number of photons (or photoelectrons) detected in a given time period. This uncertainty is dependent on the amount of signal photoelectrons being measured and has the statistical property of a Poisson distribution. This relationship is expressed as:

$$ \mathrm {\text {Shot Noise}} = \mathrm { \sqrt {\mathstrut \text {Signal} } } $$

While shot noise increases with signal, it increases more slowly (as the square root). This results in SNR improving with light levels. At low light levels, SNR is low even with a perfectly acquired image.

Signal Level (e-) Shot Noise (e- RMS) Percent of Signal
5 2.23 44.8%
10 3.16 31.6%
50 7.07 14.1%
100 10 10%
500 22.36 4.5%
1000 31.62 3.2%
Table 1.

At these lower signal levels, there have been only a few ways to improve SNR, each with a tradeoff.

  • Increase the exposure time and collect signal for a longer time
    This allows for a higher signal level, reducing the impact of shot noise. The ability to image at a desired frame rate may be sacrificed, and the cell is illuminated for a longer time, increasing phototoxicity and photobleaching. Finally, if the exposure time is long enough, the noise from dark current can become a larger portion of the signal.
  • Average frames to reduce noise
    This allows for a reduction in total image noise as a square root of the number of frames averaged. The ability to image at adequate frame rates will again be sacrificed, and is generally less productive than simply increasing exposure time.
  • Increase the excitation intensity
    This allows for a higher signal level without trading off temporal resolution. The rate at which phototoxicity and photobleaching occurs is also increased, reducing cell viability.

A remaining technique for the reduction of noise is the use of a “denoising algorithm” that dynamically examines the image collected in order to separate and remove noise. The Prime family of cameras from Photometrics introduces a new real-time method for dynamic noise reduction called PrimeEnhance.

Prime-Enhance

There are many challenges when processing data to reduce noise such as preserving the quantitative nature of the recorded pixel intensities, as well as preserving key features like edges, textures and details with low contrast. Further, processing has to be accomplished without introducing new image artifacts like ringing, aliasing or blurring. Many algorithms are inflexible with different image types, resulting in these intrusive artifacts. Additionally, because noise tends to vary with the level of signal, it is difficult for many denoising algorithms to distinguish signal from noise, and as a consequence, small details tend to be removed.

Using an algorithm invented at INRIA and optimzed for fluorescence microscopy in collaboration with the Institute Curie, PrimeEnhance implements a 2D denoising process which evaluates and processes incoming images to reduce the effects of photon shot noise at low signal levels. The algorithm also preserves the finer details and features of biological samples, and does not introduce image artifacts. One key facet of PrimeEnhance is the quantitative nature of the algorithm, ensuring that intensity values remain unchanged.

PrimeEnhance works by being aware of each camera’s characteristics and specifications. It uses this knowledge to first evaluate the image data and perform a variance stabilization transform, which removes the dependency between the mean intensities and their noise characteristics. Then a small patch of pixels is compared to similar sized patches in iteratively increasing areas of surrounding pixels (neighborhoods). The pixels within the neighborhood are selectively weighted based on their similarity to the intensity values of the original patch, and using these weighted corrections, the original patch is updated. This process is repeated through the entire image, updating each patch and reducing the impact of shot noise. Once this process has been completed, the inverse variance stabilizing transform is applied to ensure that the quantitative nature of the pixel values is maintained.

Prime-Enhance Evaluation

Fluorescence images were acquired with (Fig 1a) and without (Fig 1b) PrimeEnhance enabled, to demonstrate its functionality and give a proper comparison. The following image was acquired with a 100ms exposure time. The image statistics are avaiable in Table 2.

Figure 1a - Original Image
Figure 1a. Original Image
Figure 1b - PrimeEnhance
Figure 1b. PrimeEnhance
  Description Original Image PrimeEnhance
Full Image Average Intensity 131.7 131.1
St Dev 6.68 4.78
Min Value 72 71
Max Value 363 362
Region of Interest Average Intensity 138.8 138.2
St Dev 7.98 5.02
Min Value 87 88
Max Value 241 241
Table 2.

As shown by the intensities, the mean intensity values, minimum intensity value, and maximum intensity value remain essentially unchanged between the original noisy image and the denoised image – ensuring that all measurements made remain quantitative and are relatable to each other. The standard deviation has been reduced, indicating the removal of noise.

A difference image (Fig 1c) between the original and PrimeEnhance image shows that only noise has been removed by PrimeEnhance, with the brighter regions showing higher noise levels in keeping with the relationship discussed in the introduction.

Figure 1c - Difference Image
Figure 1c. Difference Image

Figure 2a and 2b provide an increased zoom level on the structures within the cell, and show that features are preserved while no artifacts have been generated. The line profiles demonstrate PrimeEnhance’s ability to reduce the shot noise present in the image, extracting features that were previously undistinguishable from the noise.

Figure 2a - Original Image
Figure 2a. Original Image

Figure 2c - Line Profile for Original Image
Figure 2c. Line Profile for Original Image
Figure 2b - PrimeEnhance
Figure 2b. PrimeEnhance

Figure 2d - Line Profile for PrimeEnhance
Figure 2d. Line Profile for PrimeEnhance

Prime-Enhance Experimental Impact

By increasing the effective signal to noise in each frame, it is possible to acquire high quality images at lower exposure times, reducing the effects of phototoxicity and photobleaching on samples. The following are images acquired of a faint samples with a 100ms exposure compared to images acquired with an 800ms exposure.

The comparison between the 800ms exposure and the 100ms PrimeEnhance exposure, as evidenced by the line-profiles, demonstrates the increase of image and data quality possible with PrimeEnhance at 8X lower exposure times.

Figure 3a - Raw Image at 100ms exposure
Figure 3a. Raw Image at 100ms exposure

Figure 3d - Line Profile for Raw Image at 100ms exposure
Figure 3d. Line Profile for Raw Image at 100ms exposure
Figure 3b - PrimeEnhance at 100ms exposure
Figure 3b. PrimeEnhance at 100ms exposure

Figure 3e - Line Profile for PrimeEnhance at 100ms exposure
Figure 3e. Line Profile for PrimeEnhance at 100ms exposure
Figure 3c - Raw Image at 800ms exposure
Figure 3c. Raw Image at 800ms exposure

Figure 3f - Line Profile for Raw Image at 800ms exposure
Figure 3f. Line Profile for Raw Image at 800ms exposure

Conclusion

PrimeEnhance provides a real-time quantitative increase in signal to noise ratio by reducing the effects of photon shot noise at low light levels, which improves the quality of images and data. The finer features within images are preserved and no unwanted processing artifacts are generated. A comparison between a 100ms denoised image and a 800ms standard fluorescence image shows equivalent results in image quality, indicating the ability to significantly reduce exposure times while maintaining the quality of captured data.

References

Patch-based nonlocal functional for denoising fluorescence microscopy image sequences. Boulanger J, Kervrann C, Bouthemy P, Elbau P, Sibarita JB, Salamero J. IEEE Trans. Med Imaging 2010 Feb.