Difference between revisions of "Journal:A multi-service data management platform for scientific oceanographic products"

From LIMSWiki
Jump to navigationJump to search
(Saving and adding more.)
(Saving and adding more.)
Line 38: Line 38:
==Related work==
==Related work==
The need for marine and oceanic data management supporting the SSA and the operational oceanography has led to the definition and development of various platforms that provide different types of data and services. In this context, the MyOcean project<ref name="CopernicusMEMS">{{cite web |url=http://marine.copernicus.eu/ |title=Copernicus Marine Environment Monitoring Service |publisher=European Commission |accessdate=30 January 2017}}</ref><ref name="BahurelOcean09">{{cite journal |title=Ocean Monitoring and Forecasting Core Services, the European MyOcean Example |journal=Proceedings of OceanObs'09: Sustained Ocean Observations and Information for Society |author=Bahurel, P.; Adragna, F.; Bell, M.J. et al. |volume=2009 |year=2009 |doi=10.5270/OceanObs09.pp.02}}</ref> can be mentioned, as this project implementation was in line with the best practices of the GMES/Copernicus framework.<ref name="Copernicus">{{cite web |url=http://www.copernicus.eu/ |title=Copernicus |publisher=European Commission |accessdate=30 January 2017}}</ref>
The need for marine and oceanic data management supporting the SSA and the operational oceanography has led to the definition and development of various platforms that provide different types of data and services. In this context, the MyOcean project<ref name="CopernicusMEMS">{{cite web |url=http://marine.copernicus.eu/ |title=Copernicus Marine Environment Monitoring Service |publisher=European Commission |accessdate=30 January 2017}}</ref><ref name="BahurelOcean09">{{cite journal |title=Ocean Monitoring and Forecasting Core Services, the European MyOcean Example |journal=Proceedings of OceanObs'09: Sustained Ocean Observations and Information for Society |author=Bahurel, P.; Adragna, F.; Bell, M.J. et al. |volume=2009 |year=2009 |doi=10.5270/OceanObs09.pp.02}}</ref> can be mentioned, as this project implementation was in line with the best practices of the GMES/Copernicus framework.<ref name="Copernicus">{{cite web |url=http://www.copernicus.eu/ |title=Copernicus |publisher=European Commission |accessdate=30 January 2017}}</ref>
Specifically, regarding data access, MyOcean provides the scientific community with a unified interface designed to take into account various international standards (ISO 19115<ref name="ISO19115:2003">{{cite web |url=https://www.iso.org/standard/26020.html |title=ISO 19115:2003 Geographic information -- Metadata |author=ISO/TC 211 Geographic information/Geomatics |publisher=International Organization for Standardization |date=May 2003}}</ref>, ISO 19139<ref name="ISO19139:2007">{{cite web |url=https://www.iso.org/standard/32557.html |title=ISO/TS 19139:2007 Geographic information -- Metadata -- XML schema implementation |author=ISO/TC 211 Geographic information/Geomatics |publisher=International Organization for Standardization |date=April 2007}}</ref> and INSPIRE<ref name="INSPIRE">{{cite web |url=http://inspire.ec.europa.eu/ |title=INSPIRE: Infrastructure for Spatial Information in Europe |publisher=European Commission |accessdate=30 January 2017}}</ref><ref name="EuropeanDirective07">{{cite journal |title=Directive 2007/2/EC of the European Parliament and of the Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community (INSPIRE) |journal=Official Journal of the European Union |author=European Parliament |volume=L108 |issue=25.4.2007 |year=2007 |url=http://data.europa.eu/eli/dir/2007/2/oj}}</ref>). Concerning data management, MyOcean relies on OPeNDAP/THREDDS for tasks like map subsetting and FTP for direct download. In addition, a valid solution is represented by the Earth System Grid Federation (ESGF)<ref name="CinquiniTheEarth14">{{cite journal |title=The Earth System Grid Federation: An open infrastructure for access to distributed geospatial data |journal=Future Generation Computer Systems |author=Cinquini, L.; Crichton, D.; Mattmann, C. et al. |volume=36 |issue=July 2014 |pages=400–417 |year=2014 |doi=10.1016/j.future.2013.07.002}}</ref><ref name="ESGF">{{cite web |url=https://esgf.llnl.gov/ |title=Earth System Grid Federation |publisher=ESGF Executive Committee |accessdate=30 January 2017}}</ref>, a federated system used as metadata service with advanced features, that will be described later. EMODNET MEDSEA Checkpoint<ref name="EMODnet">{{cite web |url=http://www.emodnet-mediterranean.eu/ |title=EMODnet: European Marine Observation and Data Network |publisher=European Commission |accessdate=30 January 2017}}</ref><ref name="MoussatEMODnet16">{{cite journal |title=EMODnet MedSea Checkpoint for sustainable Blue Growth |journal=Geophysical Research Abstracts |author=Moussat, E.; Pinardi, N.; Manzella, G. et al. |volume=18 |pages=9103 |year=2016 |url=http://meetingorganizer.copernicus.org/EGU2016/EGU2016-9103.pdf}}</ref> is another solution supporting data collection and data search and discovery: it exploits a checkpoint browser and a checkpoint dashboard, which presents indicators automatically produced from information databases. SeaDataNet<ref name="SeaDataNet">{{cite web |url=https://www.seadatanet.org/ |title=SeaDataNet: Pan-European Infrastructure for Ocean & Marine Data Management |publisher=European Commission |accessdate=30 January 2017}}</ref> represents a distributed marine data management infrastructure able to manage different large datasets related to in situ and remote observation of the seas and oceans. Through a distributed network approach, it provides an integrated overview and access to datasets provided by 90 national oceanographic and marine data centers. Finally, another efficient solution has been developed within the project CLIPC.<ref name="CLIPC">{{cite web |url=http://www.clipc.eu/home |title=CLIPC: Climate Information Portal |publisher=European Commission |accessdate=30 January 2017}}</ref> It provides access to climate information, including data from satellite and in situ observations, transformed data products and climate change impact indicators. Moreover, users can exploit the CLIPC toolbox to generate, compare, manipulate and combine indicators, create a user basket or launch new jobs for indices calculation.


==References==
==References==
Line 43: Line 46:


==Notes==
==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. References in the original work were alphabetical; this website organizes references in order of appearance, by design. Grammar was cleaned up in several places. A URL to Unidata's NetCDF page was originally incorrect and updated for this version.
This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. References in the original work were alphabetical; this website organizes references in order of appearance, by design. Grammar was cleaned up in several places. A URL to Unidata's NetCDF page was originally incorrect and updated for this version. URLs were added to the ISO and other standards references for improved reference information.


<!--Place all category tags here-->
<!--Place all category tags here-->

Revision as of 19:25, 25 April 2017

Full article title A multi-service data management platform for scientific oceanographic products
Journal Natural Hazards and Earth System Sciences
Author(s) D'Anca, Alessandro; Conte, Laura; Nassisi, Paola; Palazzo, Cosimo; Lecci, Rita; Cretì, Sergio; Mancini, Marco;
Nuzzo, Alessandra; Mirto, Maria; Mannarini, Gianandrea; Coppini, Giovanni; Fiore, Sandro; Aloisio, Giovanni
Author affiliation(s) Centro Euro-Mediterraneo sui Cambiamenti Climatici (CMCC), University of Salento
Primary contact Email: alessandro dot danca at cmcc dot it
Year published 2017
Volume and issue 17(2)
Page(s) 171-184
DOI 10.5194/nhess-17-171-2017
ISSN 1684-9981
Distribution license Creative Commons Attribution 3.0
Website http://www.nat-hazards-earth-syst-sci.net/17/171/2017/nhess-17-171-2017.html
Download http://www.nat-hazards-earth-syst-sci.net/17/171/2017/nhess-17-171-2017.pdf (PDF)
This article should not be considered complete until this message box has been removed. This is a work in progress.

Abstract

An efficient, secure and interoperable data platform solution has been developed in the TESSA project to provide fast navigation and access to the data stored in the data archive, as well as a standard-based metadata management support. The platform mainly targets scientific users and the situational sea awareness high-level services such as the decision support systems (DSS). These datasets are accessible through the following three main components: the Data Access Service (DAS), the Metadata Service and the Complex Data Analysis Module (CDAM). The DAS allows access to data stored in the archive by providing interfaces for different protocols and services for downloading, variable selection, data subsetting or map generation. Metadata Service is the heart of the information system of TESSA products and completes the overall infrastructure for data and metadata management. This component enables data search and discovery and addresses interoperability by exploiting widely adopted standards for geospatial data. Finally, the CDAM represents the back end of the TESSA DSS by performing on-demand complex data analysis tasks.

Introduction

TESSA (Development of TEchnology for Situational Sea Awareness) is a research project born from the collaboration between operational oceanography research and scientific computing groups in order to strengthen operational oceanography capabilities in Southern Italy for use by end users in the maritime, tourism and environmental protection sectors. This project has been very innovative as it has provided the integration of marine and ocean forecasts and analyses with advanced technological platforms. Specifically, an efficient, secure and interoperable data platform solution has been developed to provide fast navigation and access to the data stored in the data archive, as well as standard-based metadata management support.

This platform mainly targets scientific users and contains a set of high-level services such as the decision support systems (DSS) for supporting end users in managing emergency situations due to natural hazards in the Mediterranean Sea. For example, the DSS WITOIL[1][2] is crucial for oil spill accidents which could have severe impacts on the Mediterranean Basin and contribute effectively to the reduction of natural disaster risks. Moreover, the DSS Ocean-SAR[3] supports the search-and-rescue (SAR) operations following accidents, and EarlyWarning manages alerts in cases of extreme events by providing near-real-time information on weather and oceanographic conditions. Finally, VISIR[4][5] is able to compute the optimal ship route with the aim of increasing safety and efficiency of navigation. It relies on various forecast environmental fields which affect the vessel's stability in order to ensure that the computed route does not result in an exposure to dynamical hazards. In this context, the developed platform faces the lack of an efficient dissemination of marine environmental data in order to support situational sea awareness (SSA), which is strategically important for maritime safety and security.[6] In fact, an updated situation awareness requires an advanced technological system to make data available for decision makers, improving the capacity of intervention and avoiding potential damages.

In a "data-centric" perspective — in which different services, applications or users make use of the outputs of regional or global numerical models — the TESSA data platform meets the request of near-real-time access to heterogeneous data with different accuracy, resolution or degrees of aggregation. The design phase has been driven by multiple needs that the developed solution had to satisfy. First of all, nowadays each data management system should satisfy the FAIR guiding principles for scientific data management and stewardship[7]: they face the lack of a shared, comprehensive, broadly applicable and articulated guideline concerning the publication of scientific data. Specifically, they identify findability, accessibility, interoperability and reusability as the four principles which need to be addressed by data (and associated metadata) to be considered a good publication. As explained later, the employment of well-known standards concerning protocols, services and output formats satisfies these requirements. In addition, the need for a service that provides information about sea conditions 24/7 at high and very high spatial and temporal resolution has been addressed by exploiting high-performance and high-availability hardware and software solutions. The developed platform must be able to support the requests of intermediate and common users. To this end, data must be available in the native and standard format (NetCDF)[8][9] as output of the oceanographic models, through a simple and intuitive platform suitable for machine-based interactions, in order to feed user-friendly services for displaying clear maps and graphs. At the end, the system has to provide on-demand services to support decisions; the users must be able to interact with the datasets produced by the models in near-real-time. As such, the platform has to provide services and datasets suitable for on-demand processing while minimizing the downloading time and the related input file size.

This paper is organized as follows: The next section presents a survey on related work, and in the following section "The data platform architecture" an architectural overview of the main data platform components is provided. The focus is on providing easier and unified access to the heterogeneous data produced in the framework of the TESSA project. The implemented data archive and the modules that make the datasets available to the entire set of services are presented in "The Data Access Service." In "The Metadata Service and the metadata profile," the Metadata Service features are detailed from a methodological and technical point-of-view. The module developed in order to serve remote DSS submission requests is described in "The Complex Data Analysis Module." Finally in the final section on operational activity, a few use cases are presented, highlighting the operational chains which exploit the data platform services.

Related work

The need for marine and oceanic data management supporting the SSA and the operational oceanography has led to the definition and development of various platforms that provide different types of data and services. In this context, the MyOcean project[10][11] can be mentioned, as this project implementation was in line with the best practices of the GMES/Copernicus framework.[12]

Specifically, regarding data access, MyOcean provides the scientific community with a unified interface designed to take into account various international standards (ISO 19115[13], ISO 19139[14] and INSPIRE[15][16]). Concerning data management, MyOcean relies on OPeNDAP/THREDDS for tasks like map subsetting and FTP for direct download. In addition, a valid solution is represented by the Earth System Grid Federation (ESGF)[17][18], a federated system used as metadata service with advanced features, that will be described later. EMODNET MEDSEA Checkpoint[19][20] is another solution supporting data collection and data search and discovery: it exploits a checkpoint browser and a checkpoint dashboard, which presents indicators automatically produced from information databases. SeaDataNet[21] represents a distributed marine data management infrastructure able to manage different large datasets related to in situ and remote observation of the seas and oceans. Through a distributed network approach, it provides an integrated overview and access to datasets provided by 90 national oceanographic and marine data centers. Finally, another efficient solution has been developed within the project CLIPC.[22] It provides access to climate information, including data from satellite and in situ observations, transformed data products and climate change impact indicators. Moreover, users can exploit the CLIPC toolbox to generate, compare, manipulate and combine indicators, create a user basket or launch new jobs for indices calculation.


References

  1. ↑ De Dominicis, M.; Pinardi, N.; Zodiatis, G.; Lardner, R. (2013). "MEDSLIK-II, a Lagrangian marine surface oil spill model for short-term forecasting – Part 1: Theory". Geoscientific Model Development 6 (6): 1851-1869. doi:10.5194/gmd-6-1851-2013. 
  2. ↑ De Dominicis, M.; Pinardi, N.; Zodiatis, G.; Archetti, R. (2013). "MEDSLIK-II, a Lagrangian marine surface oil spill model for short-term forecasting – Part 2: Numerical simulations and validations". Geoscientific Model Development 6 (6): 1871-1888. doi:10.5194/gmd-6-1871-2013. 
  3. ↑ Coppini, G.; Jansen, E.; Turrisi, G. et al. (2016). "A new search-and-rescue service in the Mediterranean Sea: A demonstration of the operational capability and an evaluation of its performance using real case scenarios". Natural Hazards and Earth System Sciences 16 (12): 2713-2727. doi:10.5194/nhess-16-2713-2016. 
  4. ↑ Mannarini, G.; Pinardi, N.; Cippini, G. et al. (2016). "VISIR-I: Small vessels – Least-time nautical routes using wave forecasts". Geoscientific Model Development 9 (4): 1597-1625. doi:10.5194/gmd-9-1597-2016. 
  5. ↑ Mannarini, G.; Turrisi, G.; D'Anca, A. et al. (2016). "VISIR: Technological infrastructure of an operational service for safe and efficient navigation in the Mediterranean Sea". Natural Hazards and Earth System Sciences 16 (8): 1791-1806. doi:10.5194/nhess-16-1791-2016. 
  6. ↑ Lecci, R.; Coppini, G.; Creti, S. et al. (2015). "SeaConditions: Present and future sea conditions for safer navigation". Proceedings from OCEANS 2015 - Genova 2015. doi:10.1109/OCEANS-Genova.2015.7271764. 
  7. ↑ Wilkinson, M.D.; Dumontier, M.; Aalbersberg, I.J. et al. (2016). "The FAIR Guiding Principles for scientific data management and stewardship". Scientific Data 3: 160018. doi:10.1038/sdata.2016.18. PMC PMC4792175. PMID 26978244. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4792175. 
  8. ↑ "OGC network Common Data Form (netCDF) standards suite". Open Geospatial Consortium. http://www.opengeospatial.org/standards/netcdf. Retrieved 30 January 2017. 
  9. ↑ "Network Common Data Form (NetCDF)". Unidata. University Corporation for Atmospheric Research. https://www.unidata.ucar.edu/software/netcdf/. Retrieved 30 January 2017. 
  10. ↑ "Copernicus Marine Environment Monitoring Service". European Commission. http://marine.copernicus.eu/. Retrieved 30 January 2017. 
  11. ↑ Bahurel, P.; Adragna, F.; Bell, M.J. et al. (2009). "Ocean Monitoring and Forecasting Core Services, the European MyOcean Example". Proceedings of OceanObs'09: Sustained Ocean Observations and Information for Society 2009. doi:10.5270/OceanObs09.pp.02. 
  12. ↑ "Copernicus". European Commission. http://www.copernicus.eu/. Retrieved 30 January 2017. 
  13. ↑ ISO/TC 211 Geographic information/Geomatics (May 2003). "ISO 19115:2003 Geographic information -- Metadata". International Organization for Standardization. https://www.iso.org/standard/26020.html. 
  14. ↑ ISO/TC 211 Geographic information/Geomatics (April 2007). "ISO/TS 19139:2007 Geographic information -- Metadata -- XML schema implementation". International Organization for Standardization. https://www.iso.org/standard/32557.html. 
  15. ↑ "INSPIRE: Infrastructure for Spatial Information in Europe". European Commission. http://inspire.ec.europa.eu/. Retrieved 30 January 2017. 
  16. ↑ European Parliament (2007). "Directive 2007/2/EC of the European Parliament and of the Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community (INSPIRE)". Official Journal of the European Union L108 (25.4.2007). http://data.europa.eu/eli/dir/2007/2/oj. 
  17. ↑ Cinquini, L.; Crichton, D.; Mattmann, C. et al. (2014). "The Earth System Grid Federation: An open infrastructure for access to distributed geospatial data". Future Generation Computer Systems 36 (July 2014): 400–417. doi:10.1016/j.future.2013.07.002. 
  18. ↑ "Earth System Grid Federation". ESGF Executive Committee. https://esgf.llnl.gov/. Retrieved 30 January 2017. 
  19. ↑ "EMODnet: European Marine Observation and Data Network". European Commission. http://www.emodnet-mediterranean.eu/. Retrieved 30 January 2017. 
  20. ↑ Moussat, E.; Pinardi, N.; Manzella, G. et al. (2016). "EMODnet MedSea Checkpoint for sustainable Blue Growth". Geophysical Research Abstracts 18: 9103. http://meetingorganizer.copernicus.org/EGU2016/EGU2016-9103.pdf. 
  21. ↑ "SeaDataNet: Pan-European Infrastructure for Ocean & Marine Data Management". European Commission. https://www.seadatanet.org/. Retrieved 30 January 2017. 
  22. ↑ "CLIPC: Climate Information Portal". European Commission. http://www.clipc.eu/home. Retrieved 30 January 2017. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. References in the original work were alphabetical; this website organizes references in order of appearance, by design. Grammar was cleaned up in several places. A URL to Unidata's NetCDF page was originally incorrect and updated for this version. URLs were added to the ISO and other standards references for improved reference information.

Retrieved from "https://www.limswiki.org/index.php?title=Journal:A_multi-service_data_management_platform_for_scientific_oceanographic_products&oldid=30273"