PROJECT TITLE :

DPODv2 Dense Correspondence-Based 6 DoF Pose Estimation

ABSTRACT:

Using dense correspondences as the foundation, we present a three-stage, six-degrees-of-freedom object detection method that we call DPODv2 (Dense Pose Object Detector). We estimate a full 6 degrees of freedom (DoF) pose by combining a 2D object detector with a dense correspondence estimation network and a method for multi-view pose refinement. We propose a unified deep learning network, as opposed to other deep learning methods, which are typically limited to using only monocular RGB images. This network will allow for the utilization of a variety of imaging modalities (RGB or Depth). In addition to this, we suggest an innovative method for pose refinement that is predicated on differentiable rendering. In order to obtain a pose that is consistent with predicted correspondences in all views, the primary idea is to compare the predicted correspondences with the rendered correspondences in a number of different views. Our method is tested in a controlled environment using a variety of data modalities and kinds of training data, and the results are thoroughly evaluated. The most important takeaway from this study is that RGB is superior in the estimation of correspondences, whereas depth contributes to pose accuracy provided that good 3D-3D correspondences are available. Naturally, the result of their collaboration is the best possible performance overall. For the purpose of analyzing and validating the results on a number of difficult datasets, we carry out a comprehensive evaluation as well as an ablation study. DPODv2 is able to achieve excellent results on all of them while still maintaining its speed and scalability regardless of the data modality that was used or the kind of training data that was used.