PROJECT TITLE :

Representative Task Self-Selection for Flexible Clustered Lifelong Learning

ABSTRACT:

Take for example the lifelong machine learning paradigm, the goal of which is to learn a series of tasks dependent on previous experiences, such as a knowledge library or deep network weights. Consider that this paradigm. However, the knowledge libraries or deep networks for the majority of recent lifelong learning models are of a prescribed size, and this can degenerate the performance for both previously learned tasks and upcoming ones when confronted with a new task environment (cluster). We propose a novel incremental clustered lifelong learning framework that we call Flexible Clustered Lifelong Learning (FCL 3). This framework consists of two knowledge libraries: the feature learning library and the model knowledge library. The purpose of this framework is to address this challenge. To be more specific, the feature learning library that is modeled by an autoencoder architecture keeps a set of representations that are common to all of the observed tasks, and the model knowledge library can be self-selected by identifying and adding new representative models to it (clusters). When a new task is received, our FCL 3 model will first transfer knowledge from these libraries in order to encode the new task. This can be thought of as effectively and selectively soft-assigning this new task to multiple representative models based on the feature learning library. Then, either 1) the new task that has a lower outlier probability will only refine the feature learning library, or 2) the new task that has a higher outlier probability will be judged as a new representative and used to redefine both the feature learning library and the representative models over time. In order to optimize the model, we recast this problem of lifelong learning as one of alternating direction minimization. This is done whenever a new task is introduced. Finally, we evaluate the proposed framework by conducting an analysis of several different multitask data sets. The results of the experiments show that our FCL 3 model is capable of achieving better performance than the majority of lifelong learning frameworks, including batch clustered multitask learning models.