Regularizing Text Categorization with Clusters of Words

Abstract

Regularization is a critical step in any supervised learning problem and crucial for addressing not only overfitting, but also taking into account any prior knowledge we may have on the problem features and their relationships. In this paper we explore state-of-the-art structured regularizers for textual data and we propose novel ones based on topics from LSI and clusters from word2vec and graph-of-words document representation. We show that for text categorization our proposed regularizers are faster than the state-of-the-art ones while they improve classification accuracy.