PROJECT TITLE :

Light Field Spatial Super-Resolution Using Deep Efficient Spatial-Angular Separable Convolution

ABSTRACT:

Light field (LF) photography is a new method for shooting images that provide the viewer a more in-depth experience of the scene. Commercial micro-lens-based LF cameras, however, have a substantial limitation in spatial resolution because of the intrinsic tradeoff between the angular and spatial dimensions. The methods we present in this research for spatially super-resolving LF photos use end-to-end convolutional neural network models, which are both effective and efficient. With their hourglass-shaped design, the new models can extract features at a low resolution, which saves on computational and memory resources. We propose to employ 4D convolution to define the relationship between pixels in the spatial and angular domains to take full advantage of the 4D structural information in LF data. The proposed SAS convolutions are an approximation of 4D convolution, and they are more computationally and memory economical for extracting spatial-angular joint features than 4D convolution. On 57 test LF photos with varied demanding natural situations, the suggested models outperform the current state-of-the-art approaches significantly. In other words, we have achieved an average PSNR improvement of more than 3.0 dB and improved visual quality, and our approaches better preserve the LF structure of the super-resolved LF images, which is extremely desirable for following applications. A three-fold increase in speed and a small drop in reconstruction quality are both possible with the SAS convolution-based approach. Our method's source code can be found online.