This title appears in the Scientific Report :
2020
Please use the identifier:
http://hdl.handle.net/2128/24858 in citations.
Performance analysis and optimization of a TByte-scale atmospheric observation database
Performance analysis and optimization of a TByte-scale atmospheric observation database
We present performance engineering of a TByte-scale air quality database (DB) system which was created by the Tropospheric Ozone Assessment Report (TOAR) and contains one of the world’s largest collections of near-surface air quality measurements. A special feature of our data service https://join.f...
Saved in:
Personal Name(s): | Betancourt, Clara (Corresponding author) |
---|---|
Hagemeier, Björn / Schröder, Sabine / Schultz, Martin | |
Contributing Institute: |
Jülich Supercomputing Center; JSC |
Imprint: |
2020
|
Conference: | EGU2020: Sharing Geoscience Online, online (online conference), 2020-05-04 - 2020-05-08 |
Document Type: |
Conference Presentation |
Research Program: |
Earth System Data Exploration Doktorand ohne besondere Förderung Artificial Intelligence for Air Quality Data-Intensive Science and Federated Computing |
Link: |
OpenAccess OpenAccess |
Publikationsportal JuSER |
We present performance engineering of a TByte-scale air quality database (DB) system which was created by the Tropospheric Ozone Assessment Report (TOAR) and contains one of the world’s largest collections of near-surface air quality measurements. A special feature of our data service https://join.fz-juelich.deis on-demand processing of several air quality metrics directly from the TOAR database. As a service that is used by more than 150 users of the international air quality research community, our web service must be easily accessible and functionally flexible, while delivering good performance. The current on-demand calculations of air quality metrics outside the database are identified as the major performance bottleneck. In this study, we therefore explore and benchmark in-database approaches for the statistical processing, which result in performance enhancements of up to 32%. We will furthermore show how the web service infrastructure can be extended in functionality, allowing the calculation of flux-based ozone metrics. |