DSpace
 

Researchspace >
General science, engineering & technology >
General science, engineering & technology >
General science, engineering & technology >

Please use this identifier to cite or link to this item: http://hdl.handle.net/10204/4091

Title: Comparison of data file and storage configurations for efficient temporal access of satellite image data
Authors: Bachoo, A
Van den Bergh, F
Gazendam, A
Keywords: Hierarchical data format
Network common data format
netCDF
Satellite image data
HDF5
Satellite time series storage
Issue Date: 2009
Citation: Bachoo, A, Van den Bergh, F and Gazendam, A. 2009. Comparison of data file and storage configurations for efficient temporal access of satellite image data. South African Computer Journal, Vol. 43, pp 66-74
Abstract: Satellite data volumes have seen a steady increase in recent years due to improvements in sensor technology and increases in data acquisition frequency. The gridded MODIS data products, spanning a region of interest of approximately 10° by 10° for a single title, are stored as images containing almost six million pixels, with data in multiple spectral bands for each pixel. Time series analyses of a sequence of such images in order to perform automated change detection is a topic of growing importance. Traditional storage formats store such a series of images as a sequence of individual files, with each file internally storing the pixels in their spatial order. Consequently, the construction of a time series profile of a single pixel requires reading from several hundred large files, resulting in substantial performance overheads that severely constrain high-throughput analyses. We aim to minimize this performance limitation by restructuring the storage scheme for typical satellite imagery as temporal sequences in order to reduce overheads and improve throughput. Models are developed to compute the expected query time for both the time-sequential and the traditional image-based representations. These models are used to demonstrate the benefits of using a time-sequential representation. Four data structures (using the Hierarchical Data Format (HDF5), Network Common Data Format (netCDF) and a native file system approach) are implemented and compared in a series of experimental read tests to determine which format is most appropriate for implementation in the CSIR Cluster Computing Centre's facilities.
URI: http://foss4g.org/index.php/foss4g/2008/paper/viewFile/140/57
http://hdl.handle.net/10204/4091
ISBN: 978-0-620-42117-1
Appears in Collections:ICT for earth observation
General science, engineering & technology

Files in This Item:

File Description SizeFormat
Bachoo1_2009.pdf232.02 kBAdobe PDFView/Open
View Statistics

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

 

Valid XHTML 1.0! DSpace Software Copyright © 2002-2010  Duraspace - Feedback