Topic modelling of news articles for two consecutive elections in South Africa

Abstract

In election cycles, the political-themed articles published by news providers present a rich source of information about election discourse. Extracting useful themes from a large article corpus manually is infeasible, text mining techniques such as topic modelling provide a mechanism to automatically infer themes from a corpus of text. Exploring the coverage of a single election period uncovers topical discourse that is relevant to current affairs in that election period. Analysing two consecutive election periods allows one to analyse the evolution of discourse from one period to another. Articles published by News24 were sourced to conduct the analysis and answer the research questions set forth. The articles were cleaned and topic models were built to identify 20 latent topics. The articles are classified with their topic before a pairwise cosine similarity comparison is applied on topic corpora to identify similar topics between election periods. The results of this study provide important insights relating to the two election periods, some of these include: coverage of corruption related content is consistent between the two election periods and most political-themed articles in this corpus address problematic themes.