GEOBIG. Open data big volumes management

Abstract

The available geographic information is growing by leaps and
bounds. New data is published every day, often through open
data portals, which means investments that are not negligible for
its creation and maintenance. However, they are often difficult
to reuse, given their heterogeneity and the variety of forms of
access.
At the same time, technology related to artificial intelligence has
undergone a great advance, however, it is difficult to apply these
technologies to large open data sets, due to their dispersion and
the difficulties in processing them. Therefore, it is necessary to
prepare the data prior to any analysis process.
It will be exposed how more than 125,000 layers from all over the
world have been preprocessed, containing more than 250 million
elements, and how they have been prepared and integrated into
an architecture capable of applying processes in an automated
way, regardless of their characteristics.
Aspects such as identification and access to data, and the processes
followed until obtaining a system capable of processing
thousands of layers in parallel, such as enrichment of metadata
are exposed. The system allows to obtain the maximum value
of the huge amount of geographic data available, whether they
were INSPIRE data or not.