Applying sensor web strategies to big data earth observations

Abstract

Earth observation data and meta-data are a central concern of the earth sciences. These data are generated by a myriad of both in-situ and remote sensors. Other sources of data include computational simulations, various ex-situ sources such as environmental sampling campaigns and emerging trends such as crowd sourcing. The Big Data phenomenon is one that has always existed in the earth observation community due to the large swaths of homogeneous data grids produced by for instance earth observation satellites. These data grids are continuously increasing in volume as spatial resolution and number of observed variables increases. The Sensor Web adds to this volume challenge by adding two additional V’s namely variety and velocity. Variety talks to the heterogeneity of the data available from the Sensor Web whilst velocity refers to the high temporal resolution of the data and the need for near-real time processing in order for the produced information to be relevant. The Big Data phenomenon compounds the already existing technological challenges associated with these earth observation data sets. These challenges include data discovery, data access, data exploration, data pre-processing, data analytics and data presentation. What has not been explored by the research community is how those Big-Data accessible to the end user through the Sensor Web are to be used effectively to achieve exploration, pre-processing and integration, processing and modelling and eventually presentation and visualisation. Added to this lack of know how is the additional uncertainty linked to the exact relationship between Big Data and Service oriented Architectures.