Incremental updating and versioning

Abstract

A user of geographical data invariably compiles their data base using core data sets obtained from various sources. They then integrate these together to meet their particular needs and build their own, value-added data and topology on top. The user’s concern is their value-added data and maintaining their integrity, quality and spatial referencing. However, the core data provide them with a crucial spatial framework for their value-added data. As geographical data are dynamic, most suppliers of core data sets maintain and update their data. Such updates could be continuous (eg: telemetry), periodic (eg: daily, monthly or annually), part of a planned update cycle (eg: for a national mapping series), when the amount of changes crosses a threshold, or by special request (eg: census or election). •Historically, these updates have been provided to the end user in bulk, as a new data set to replace the old one. The user could ignore the update (if it is not significant enough), use it to manually (and selectively) update their data base, or accept the whole update in toto - and have to re-integrate it with their other core data sets and rebuild their value-added data and topology. They would also have to understand how any changes, aggregations, sub-divisions, additions or deletions might affect the geocoding or referencing of their value-added data, such as through the loss or change of an unique identifier. •The user would also need to track all the different versions they have received and used, to ensure that they implement the updates in the correct order, that they do not re-implement updates or that they do not miss updates. Unlike software, a user might need to keep several different versions of the same data set and even use them together simultaneously, such as for time series analysis. •This presentation will outline this problem of incremental updating (providing updates successively to users) and versioning (keeping track of different versions of a data set), and will attempt to provide a conceptual framework of the issues, such as: •The data set’s temporal domain – the currency or validity of the data, which can be dependent on the scale of the data The cartographer’s temporal domain – the period of validity of the cartographer’s knowledge and understanding of the data; Using knowledge of the data set’s and cartographer’s temporal domains to resolve disputes and for historical research; Long transactions and parallel updates; When a change should be registered; and How incremental updating and versioning benefits the data producer.