PROJECT TITLE :

Temporal Head Pose Estimation From Point Cloud in Naturalistic Driving Conditions

ABSTRACT:

Estimating a person's head pose is an important problem because it helps with tasks like modeling attention and estimating where a person is looking. Within the realm of automobiles, the position of the driver's head can reveal important details about their mental state, such as whether or not they are drowsy, distracted, or attentive. In-car entertainment and information systems can also be interacted with using this component. Although computer vision algorithms that use RGB cameras are reliable in environments that have been carefully controlled, estimating the head's pose in a moving vehicle presents a difficult challenge. This is because sudden shifts in illumination, occlusions, and large head rotations are all commonplace in moving vehicles. Utilizing depth cameras is one method that can help alleviate some of these problems. Trajectories of head rotation are continuous, and they have significant temporal dependencies. This observation is used as a jumping off point for our research, in which we propose a novel temporal deep learning model for head pose estimation using point clouds. This method utilizes the three-dimensional spatial structure of the face in order to derive the discriminative feature representation directly from the point cloud data. After that, the frame-based representations are combined with layers of bidirectional long short term memory, also known as BLSTM. We achieve better results with this model when compared to non-temporal algorithms that use point cloud data and state-of-the-art models that use RGB images. This is because we train this model on the newly collected multimodal driver monitoring (MDM) dataset. In addition, we demonstrate both quantitatively and qualitatively that the incorporation of temporal information results in significant improvements not only in the level of accuracy but also in the degree to which the predictions are smooth.