Massive standardization and geocoding of postal addresses through ETL processes using unified digital street map of Andalusia (CDAU) web services

Abstract

The Inventory of headquarters and public services of «Junta de Andalucía» (ISE) offers a global vision of the location of the services provided by the Andalusian Regional Government, offering both the geographical location and the most relevant alphanumeric data. The ISE solves the problems associated with the great dispersion of data and formats in which each responsible body publishes them, integrating them into a single standardized PostgreSQL/PostGIS database that can be consulted through a web viewer and interoperable OGC web services. A major task of the project is to process the starting information so that the postal addresses are normalized and that each facility or headquarter is located through a geometric point. These operations (normalization and geocoding) are executed automatically using the web processing service (WPS) of the Unified Digital Street Map of Andalusia (CDAU), the official reference source for roads and portals in Andalusia. To facilitate the task, the queries have been automated within workflows implemented using the ETL Kettle tool, one of the products of the Pentaho Data Integration suite. To do this, the normalization and geocoding functions are recursively invoked through the REST communication protocol and the response returned in JSON format is interpreted, isolating the values of interest (road type, road name, gate number and set of coordinates) and generating the corresponding geometries.