Abstract: Data are the fundamental building blocks to conduct scientific studies that seek to understand natural phenomena in space and time. The notion of data processing is ubiquitous and nearly operates in any project that requires gaining insight from the data. The increasing availability of information sources, data formats and download services offered to the users, makes it difficult to reuse or exploit the potential of those new resources in multiple scientific fields. In this paper, we present a spatial extract-transform-load (spatial-ETL) approach for downloading atmospheric datasets in order to produce new biometeorological indices and expose them publicly for reuse in research studies. The technologies and processes involved in our work are clearly defined in a context where the GDAL library and the Python programming language are key elements for the development and implementation of the geoprocessing tools. Since the National Oceanic and Atmospheric Administration (NOAA) is the source of information, the ETL process is executed each time this service publishes an updated atmospheric prediction model, thus obtaining different forecasts for spatial and temporal analyses. As a result, we present a web application intended for downloading these newly created datasets after processing, and visualising interactive web maps with the outcomes resulting from a number of geoprocessing tasks. We also elaborate on all functions and technologies used for the design of those processes, with emphasis on the optimisation of the resources as implemented in cloud services.

Authorship

Download reference