PROJECT TITLE :

Continual Adaptation for Deep Stereo

ABSTRACT:

The use of convolutional neural networks that have been trained end-to-end to regress dense disparities yields results that cannot be matched when it comes to the accuracy of depth estimation from stereo images. As is the case with the vast majority of tasks, this can be accomplished if significant quantities of labelled samples are made available for training, ideally covering the entire data distribution that will be encountered during deployment. In light of the fact that such an assumption is invariably invalidated in practice, the capability of adjusting to any unfamiliar environment assumes a position of preeminent significance. For this reason, we propose a paradigm of continuous adaptation for deep stereo networks that is intended to deal with environments that are both challenging and constantly changing. We design a lightweight and modular architecture called the Modularly AD aptive Network (MADNet), and we formulate M odular AD aptation algorithms (MAD, MAD ++) that allow for the efficient optimization of independent sub-portions of the entire network. The learning signals that are required to continuously adapt models online can be derived, according to our model, either from self-supervision in the form of right-to-left image warping or from conventional stereo algorithms. Both of these methods are viable options. When using both sources, there is no requirement for any other data besides the input images that are being gathered during the deployment time. Thus, our network architecture and adaptation algorithms realize the first real-time self-adaptive deep stereo system and pave the way for a new paradigm that can facilitate the practical deployment of end-to-end architectures for dense disparity regression. In addition, our work paves the way for a new paradigm that can facilitate the practical deployment of deep learning models for object recognition.