PROJECT TITLE :

Task-Driven Comparison of Topic Models

ABSTRACT:

Topic modeling, a technique of statistically extracting thematic content from a giant assortment of texts, is employed for a wide variety of tasks inside text analysis. Though there are a growing range of tools and techniques for exploring single models, comparisons between models are generally reduced to a little set of numerical metrics. These metrics may or might not replicate a model's performance on the analyst's intended task, and can so be insufficient to diagnose what causes variations between models. During this paper, we explore task-centric topic model comparison, considering how we will each provide detail for a more nuanced understanding of differences and address the wealth of tasks for which topic models are used. We derive comparison tasks from single-model uses of topic models, which predominantly fall into the classes of understanding topics, understanding similarity, and understanding change. Finally, we provide several visualization techniques that facilitate these tasks, as well as buddy plots, that mix color and position encodings to allow analysts to readily view changes in document similarity.