A situational awareness tool using Open-Source Intelligence (OSINT) and Artificial Intelligence (AI)

Abstract

Social media platforms have become a means for users to share information, interests, and events with friends, among others. This shift has led to a surge in shared content related to events, such as posts that reflect some people's concerns about real-life occurrences. As a result, significant progress has been made in developing situational awareness systems that offer valuable insights derived from data collected from various sources, especially social media platforms. However, few studies in the literature focus on the South African landscape. Additionally, these studies are mainly based on Twitter data, overlooking the use of information shared by media publishers. Furthermore, most research analyses posts only after civil unrest has occurred. This study proposes a situational awareness tool that uses open-source intelligence (OSINT) and machine learning algorithms to predict civil unrest events. It uses data from Twitter and news media platforms such as SABC News, Eyewitness News (EWN), and News24, in addition to information from the Armed Conflict Location & Event Data Project (ACLED). We employ natural language processing (NLP) to process and explore the data to obtain insights and train supervised learning models, including logistic regression, support vector machines, decision trees, and random forest classifiers. These models are evaluated using CountVectorizer, term frequency-inverse document frequency (TF-IDF) and LabelEncoder for data normalisation. Experimental evaluations reveal that the logistic regression model, paired with TF-IDF, outperforms the other models in predicting instances of social unrest with the highest accuracy. Predicting such events helps law enforcement agencies, organisations, and individuals understand and anticipate these occurrences, allowing proactive measures to protect peace, public safety, and economic stability.