Searching for Sentinel data is often a complicated process due to the different missions available, the different Copernicus Sentinel Hubs that host data and geographic restrictions, but also the different performances of the Hubs in terms of download speed and latency (at both the inter and intra level). Thus, at BEYOND Center of EO Research and Satellite Remote Sensing we developed an Umbrella Data Hub that brings the various Sentinel Hubs all together in a single data access (the so called federated access) point offering to the users uniform access to Sentinel 1, Sentinel 2, Sentinel 3, and Sentinel 5p metadata via connecting in the back end to a number of the available Copernicus Sentinel Hubs and serving the data search results via a front end catalogue and an Application Programming Interface (API). Most of the harvested Copernicus Sentinel Hubs are using data hub system (DHuS) that allows users to access the data via their own computer programs, scripts or client applications. API Queries are based on OpenSearch protocol, which is a collection of technologies that allow publishing of search results in a format suitable for syndication and aggregation.
As this Umbrella hub saves metadata from several hubs, there is the need for a database system. The database stores not only information about the metadata, but also information about the hubs. Metadata information is the same with the one on the source hubs, e.g. polarization for Sentinel-1 or cloud coverage for Sentinel-2. Moreover, the database is populated with new data so that users have access to newly ingested metadata as fast as possible. Thus, a searching application is executed every fifteen minutes. At the same time, hubs delete their data due to either their rolling archive policy or due to several other reasons, such as duplicate, corrupted products et cetera. The Umbrella hub has to update the metadata in the database by deleting them when they are deleted from the hubs. Therefore, the delete process takes place once per day to prevent users from attempting to download a product that is either offline or deleted.
The greatest advantage of this Umbrella hub is that it provides users the download link from the most appropriate hub for the requested product. This is achieved by a scoring process which runs every ten minutes and checks the availability of the hubs. If the hub is available, it also measures its download speed. Hubs get points based on their availability status and download speed. The total score of each hub is stored in the database. Thus, when users make a request to the Umbrella hub API, they get as a response the metadata that stem from the hub with the highest score and in which the products are available. This way, users take advantage of the best combination of the product-hub pair. At the next sections there is further elaboration on these two modules.
MAIN OBJECTIVES & OUTCOMES
Ultimately, this application gives the potential for:
- Linking Copernicus Sentinels Hubs to a single data hub, instead of searching for the appropriate one for the user’s needs.
- Accessing to all Sentinel mission data and better performance on downloading products, as products are chosen from the most appropriate data hub based on integrity, speed and availability tests.
RELATED SERVICES & PLATFORMS
The Sentinel data federated gateway is accessed through the Application Programming Interface (API) directly from the Hellenic Mirror Site :https://sentinels.space.noa.gr/dhus/#/home