PROJECT TITLE :

Attention in Reasoning Dataset, Analysis, and Modeling

ABSTRACT:

Although attention has become an increasingly popular component in deep neural networks for the purpose of both interpreting data and improving the performance of models, relatively little research has been done to investigate how attention develops throughout the completion of a task and whether it is reasonable for attention to develop in this way. In this body of work, we propose an Attention with Reasoning capability (AiR) framework, which makes use of attention in order to comprehend and enhance the process that leads to task outcomes. We begin by defining an evaluation metric that is based on a series of atomic reasoning operations. This makes it possible to conduct a quantitative measurement of attention that takes into account the reasoning process. The next step is for us to collect data on human eye-tracking and answer accuracy, after which we conduct an analysis of a variety of machine and human attention mechanisms, focusing on their capacity for reasoning and how they influence task performance. We propose supervising the learning of attention progressively along the reasoning process and differentiating between correct and incorrect attention patterns in order to improve the attention and reasoning ability of visual question answering models. This will allow us to improve the attention of the models and their ability to reason. We show that the proposed framework is effective in analyzing and modeling attention, leading to improvements in both reasoning ability and task performance.