ACM Faculty Talk
Mixed Membership Word Embeddings for Computational Social Science
Dr. James Foulds, Information Systems, UMBC
12:00-1:00pm Thursday, 5 April 2018, ITE459, UMBC
Word embeddings improve the performance of natural language processing (NLP) systems by revealing the hidden structural relationships between words. Despite their success in many applications, word embeddings have seen very little use in computational social science NLP tasks, presumably due to their reliance on big data, and to a lack of interpretability. I propose a probabilistic model-based word embedding method which can recover interpretable embeddings, without big data. The key insight is to leverage mixed membership modeling, in which global representations are shared, but individual entities (i.e., dictionary words) are free to use these representations to uniquely differing degrees. I show how to train the model using a combination of state-of-the-art training techniques for word embeddings and topic models. The experimental results show an improvement in predictive language modeling of up to 63% in MRR over the skip-gram, and demonstrate that the representations are beneficial for supervised learning. I illustrate the interpretability of the models with computational social science case studies on State of the Union addresses and NIPS articles.
James (a.k.a. Jimmy) Foulds is an assistant professor in the Department of Information Systems at UMBC. His research interests are in both applied and foundational machine learning, focusing on probabilistic latent variable models and the inference algorithms to learn them from data. His work aims to promote the practice of latent variable modeling for multidisciplinary research in areas including computational social science and the digital humanities. He earned his Ph.D. in computer science at the University of California, Irvine, and was a postdoctoral scholar at the University of California, Santa Cruz, followed by the University of California, San Diego. His master’s and bachelor’s degrees were earned with first class honours at the University of Waikato, New Zealand, where he also contributed to the Weka data mining system.