PROJECT TITLE :

Short Text Topic Modeling Techniques, Applications, and Performance: A Survey

ABSTRACT:

The semantic understanding of short texts is required for a wide variety of real-world applications, so their analysis allows for the inference of distinct and consistent latent topics, which is an important and fundamental task. Because only very limited information regarding word co-occurrences is available in short texts, traditional long text topic modeling algorithms such as PLSA and LDA, which are based on word co-occurrences, are unable to solve this problem very effectively. Because of this, short text topic modeling has already attracted a lot of attention from the community of researchers who work on machine learning in the recent years. This attention is directed toward finding a solution to the problem of sparseness in short texts. In this survey, we conduct an in-depth review of the many different short text topic modeling techniques that have been proposed in the previous research. We present three categories of methods that are based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation. For each category, we provide an example of a representative approach as well as an analysis of how well these methods perform on a variety of tasks. We develop the first comprehensive open-source library for use in Java called STTM. It integrates all surveyed algorithms within a unified interface, benchmark datasets, to make it easier for new methods to be developed within this research field. Finally, we evaluate the performance of these state-of-the-art methods on a variety of real-world datasets and compare their results against both one another and a long text topic modeling algorithm.