Du Toit, KirstenFogwill, ThomasMaiwashe, TVan der Westhuizen, M2026-08-072026-08-072015-11http://hdl.handle.net/10204/14888The resource constrained environment in South Africa has led to an increased interest in the monitoring and control of electricity demand. The associated proliferation of smart meters/sensors and the collection of their data allows for monitoring and control of demand on a much wider scale than previously possible. A homeowner, business or utility may now make use of real-time data to track and control their electricity demand with potentially widespread economic effect. We examine the characteristics that a system must exhibit to excel at high volume, low latency real-time data stream processing to support such applications. A robust, scalable and fast method for real-time processing of streamed electricity demand data is suggested. This method also addresses the problems associated with developing countries in that the data received are sometimes inconsistent, unevenly spaced and interrupted. Processing these unevenly spaced data for visualisation is an important requirement in order for users to more easily understand and analyse their usage trends. Binned aggregation is explored as an effective strategy for minimizing the negative effects of inconsistent data of different frequencies. It is also used to reduce the data to manageable dimension, while real-time processing determines where and how to use the data. A sensor's data are stored in several rolling arrays of differing size that represent different time-frames. We keep the most detailed information for the current time-frame. The format chosen allows for data reduction yet allows redundant data to contribute to the model and refine the aggregation. The reduced amount of data stored allows fast retrieval of those data for visual display and analysis and improved scalability.FulltextenReal-time stream processingUnevenly spaced dataBinned aggregationVisualisationTime-seriesSensor dataConference Presentationn/a