PROJECT TITLE :

HarMI Human Activity Recognition via Multi-Modality Incremental Learning

ABSTRACT:

Human activity recognition (HAR) is a topic that has received a lot of attention in recent years due to the proliferation of different types of sensors that can be found in smartphones or wearable devices. HAR has many applications in fields such as healthcare, smart cities, and other areas. For sensor-based HAR, numerous approaches have been proposed, many of which are based on hand-crafted feature engineering or deep neural networks. However, these existing methods typically recognize offline activities, which means that all of the data should be collected before training, which occupies a large amount of storage space. Additionally, these methods are not very efficient. In addition, once the offline model training is complete, the trained model is unable to recognize new activities unless it undergoes further training from the beginning; this results in a significant increase in both the amount of time and space required. In this article, we propose a multi-modality incremental learning model with the capacity for continuous learning. We will refer to this model as HarMI. The HarMI model that has been proposed can begin training in a short amount of time using a small amount of storage space, and it can easily learn new activities without storing previous training data. In more specific terms, we first make use of an attention mechanism in order to align heterogeneous sensor data with various frequencies. In addition, HarMI employs the elastic weight consolidation and canonical correlation analysis from a multi-modality point of view in order to circumvent the catastrophic forgetting that can occur during the incremental learning process. Extensive tests conducted on two publicly available datasets have shown that HarMI is capable of achieving a performance that is superior to that of a number of other state-of-the-art methods.