PROJECT TITLE :

Physics-based Noise Modeling for Extreme Low-light Photography

ABSTRACT:

Improving one's visibility in conditions of extremely low light is a difficult task to undertake. In conditions with almost no light, the existing image denoising methods could very well fail due to the significantly low signal-to-noise ratio (SNR). In this paper, we conduct an in-depth study of the noise statistics that occur throughout the imaging pipeline of CMOS photosensors and develop a comprehensive noise model that is able to accurately characterize the various real-world noise structures. Our innovative model takes into account the noise sources that are caused by the electronics of digital cameras. These noise sources are largely ignored by the methods that are currently in use, despite the fact that they have a significant impact on raw measurements made in the dark. It offers a method for decoupling the complex noise structure into a variety of statistical distributions that can be interpreted in terms of physical phenomena. In addition to this, our noise model can be used to generate realistic training data for algorithms that are based on learning to reduce noise in low-light images. In this regard, recent research with deep convolutional neural networks has shown some encouraging results; however, the success of these networks is heavily dependent on a large number of noisy-clean image pairs for training, which are extremely challenging to acquire in actual practice. The problem also arises when they attempt to generalize their trained models to images from new devices. Extensive testing on a variety of low-light denoising datasets, including one that was specifically collected for this work and covers a wide range of devices, demonstrates that a deep neural network that has been trained using our proposed model of noise formation can achieve a level of accuracy that is surprisingly high. The results are comparable to, and in some cases even better than, training with paired real data, which paves the way for new possibilities in extreme low-light photography in the real world.