|
Eyring, V., Bock, L., Lauer, A., Righi, M.,
Schlund, M., Andela, B., Arnone, E., Bellprat, O.,
Brötz, B., Caron, L.-P., Carvalhais, N., Cionni,
I., Cortesi, N., Crezee, B., Davin, E., Davini, P.,
Debeire, K., de Mora, L., Deser, C., Docquier, D.,
Earnshaw, P., Ehbrecht, C., Gier, B. K.,
Gonzalez-Reviriego, N., Goodman, P., Hagemann, S.,
Hardiman, S., Hassler, B., Hunter, A., Kadow, C.,
Kindermann, S., Koirala, S., Koldunov, N., Lejeune,
Q., Lembo, V., Lovato, T., Lucarini, V., Massonnet,
F., Müller, B., Pandde, A., Pérez-Zanón, N.,
Phillips, A., Predoi, V., Russell, J., Sellar, A.,
Serva, F., Stacke, T., Swaminathan, R., Torralba,
V., Vegas-Regidor, J., von Hardenberg, J., Weigel,
K., and Zimmermann, K.: Earth System Model Evaluation Tool
(ESMValTool) v2.0 - an extended set of large-scale diagnostics
for quasi-operational and comprehensive evaluation of
Earth system models in CMIP, Geosci. Model Dev., 13,
3383-3438, doi: 10.5194/gmd-13-3383-2020, 2020.
|
The Earth System Model Evaluation Tool
(ESMValTool) is a community diagnostics and
performance metrics tool designed to improve
comprehensive and routine evaluation of Earth
System Models (ESMs) participating in the Coupled
Model Intercomparison Project (CMIP). It has
undergone rapid development since the first release
in 2016 and is now a well-tested tool that provides
end-to-end provenance tracking to ensure
reproducibility. It consists of (1) an easy-to-install,
well documented Python package providing the core
functionalities (ESMValCore) that performs common
pre-processing operations and (2) a diagnostic part
that includes tailored diagnostics and performance
metrics for specific scientific applications. Here
we describe large-scale diagnostics of the second
major release of the tool that supports the
evaluation of ESMs participating in CMIP Phase 6
(CMIP6). ESMValTool v2.0 includes a large
collection of diagnostics and performance metrics
for atmospheric, oceanic, and terrestrial variables
for the mean state, trends, and variability.
ESMValTool v2.0 also successfully reproduces
figures from the evaluation and projections
chapters of the Intergovernmental Panel on Climate
Change (IPCC) Fifth Assessment Report (AR5) and
incorporates updates from targeted analysis
packages, such as the NCAR Climate Variability
Diagnostics Package for the evaluation of modes of
variability the Thermodynamic Diagnostic Tool
(TheDiaTo) to evaluate the energetics of the
climate system, as well as parts of AutoAssess that
contains a mix of top-down performance metrics. The
tool has been fully integrated into the Earth
System Grid Federation (ESGF) infrastructure at the
Deutsches Klimarechenzentrum (DKRZ) to provide
evaluation results from CMIP6 model simulations
shortly after the output is published to the CMIP
archive. A result browser has been implemented that
enables advanced monitoring of the evaluation
results by a broad user community at much faster
timescales than what was possible in CMIP5.
|
|
Lauer, A., Eyring, V., Bellprat, O., Bock, L., Gier, B. K.,
Hunter, A., Lorenz, R., Pérez-Zanón, N., Righi, M., Schlund, M.,
Senftleben, D., Weigel, K., and Zechlau, S.: Earth System Model
Evaluation Tool (ESMValTool) v2.0 - diagnostics for emergent
constraints and future projections from Earth system models in
CMIP, Geosci. Model. Dev., 13, 4205-4228, doi:
10.5194/gmd-13-4205-2020, 2020.
|
The Earth System Model Evaluation Tool (ESMValTool), a community
diagnostics and performance metrics tool for evaluation and analysis
of Earth system models (ESMs) is designed to facilitate a more
comprehensive and rapid comparison of single or multiple models
participating in the Coupled Model Intercomparison Project (CMIP).
The ESM results can be compared against observations or reanalysis
data as well as against other models including predecessor versions
of the same model. The updated and extended version 2.0 of the
ESMValTool includes several new analysis scripts such as large-scale
diagnostics for evaluation of ESMs as well as diagnostics for extreme
events, regional model and impact evaluation. In this paper, the newly
implemented climate metrics such as effective climate sensitivity (ECS)
and transient climate response (TCR) as well as emergent constraints
for various climate-relevant feedbacks and diagnostics for future
projections from ESMs are described and illustrated with examples
using results from the well-established model ensemble CMIP5. The
emergent constraints implemented include constraints on ECS,
snow-albedo effect, climate-carbon cycle feedback, hydrologic cycle
intensification, future Indian summer monsoon precipitation, and year
of disappearance of summer Arctic sea ice. The diagnostics included
in ESMValTool v2.0 to analyze future climate projections from ESMs
further include analysis scripts to reproduce selected figures of
chapter 12 of the Intergovernmental Panel on Climate Change’s (IPCC)
Fifth Assessment report (AR5) and various multi-model statistics.
|
|
Righi, M., Andela, B., Eyring, V., Lauer, A., Predoi, V.,
Schlund, M., Vegas-Regidor, J., Bock, L., Brötz,
B., de Mora, L., Diblen, F., Dreyer, L., Drost,
N., Earnshaw, P., Hassler, B., Koldunov, N.,
Little, B., Loosveldt Tomas, S., and Zimmermann,
K.: Earth System Model Evaluation Tool (ESMValTool) v2.0 - technical overview,
Geosci. Model Dev., 13, 1179-1199, doi: 10.5194/gmd-13-1179-2020, 2020.
|
This paper describes the second major release of the Earth System Model
Evaluation Tool (ESMValTool), a community diagnostic and performance
metrics tool for the evaluation of Earth system models (ESMs) participating
in the Coupled Model Intercomparison Project (CMIP). Compared to version
1.0, released in 2016, ESMValTool version 2.0 (v2.0) features a brand new
design, with an improved interface and a revised preprocessor. It also
features a significantly enhanced diagnostic part that is described in
three companion papers. The new version of ESMValTool has been specifically
developed to target the increased data volume of CMIP Phase 6 (CMIP6) and the
related challenges posed by the analysis and the evaluation of output from
multiple high-resolution or complex ESMs. The new version takes advantage of
state-of-the-art computational libraries and methods to deploy an efficient
and user-friendly data processing. Common operations on the input data (such
as regridding or computation of multi-model statistics) are centralized in a
highly optimized preprocessor, which allows applying a series of preprocessing
functions before diagnostics scripts are applied for in-depth scientific analysis
of the model output. Performance tests conducted on a set of standard diagnostics
show that the new version is faster than its predecessor by about a factor of 3.
The performance can be further improved, up to a factor of more than 30, when
the newly introduced task-based parallelization options are used, which enable
the efficient exploitation of much larger computing infrastructures. ESMValTool
v2.0 also includes a revised and simplified installation procedure, the setting
of user-configurable options based on modern language formats, and high code
quality standards following the best practices for software development.
|
|
Schlund, M., Hassler, B., Lauer, A., Andela, B., Jöckel, P., Kazeroni, R.,
Loosveldt Tomas, S., Medeiros, B., Predoi, V., Sénési, S., Servonnat, J.,
Stacke, T., Vegas-Regidor, J., Zimmermann, K., and Eyring, V.:
Evaluation of Native Earth System Model Output with ESMValTool v2.6.0,
Geosci. Model Dev., 16, 315-333, doi: 10.5194/gmd-16-315-2023, 2023.
|
Earth system models (ESMs) are state-of-the-art climate models that allow
numerical simulations of the past, present-day, and future climate. To
extend our understanding of the Earth system and improve climate change
projections, the complexity of ESMs heavily increased over the last decades.
As a consequence, the amount and volume of data provided by ESMs has increased considerably.
Innovative tools for a comprehensive model evaluation and analysis are required to assess
the performance of these increasingly complex ESMs against observations or reanalyses. One
of these tools is the Earth System Model Evaluation Tool (ESMValTool), a community diagnostic
and performance metrics tool for the evaluation of ESMs. Input data for ESMValTool needs to
be formatted according to the CMOR (Climate Model Output Rewriter) standard, a process that is
usually referred to as "CMORization". While this is a quasi-standard for large model intercomparison
projects like the Coupled Model Intercomparison Project (CMIP), this complicates the application
of ESMValTool to non-CMOR-compliant climate model output.
In this paper, we describe an extension of ESMValTool introduced in v2.6.0 that allows seamless
reading and processing of "native" climate model output, i.e., operational output produced by
running the climate model through the standard workflow of the corresponding modeling institute.
This is achieved by an extension of ESMValTool's preprocessing pipeline that performs a CMOR-like
reformatting of the native model output during runtime. Thus, the rich collection of diagnostics
provided by ESMValTool is now fully available for these models. For models that use unstructured
grids, a further preprocessing step required to apply many common diagnostics is regridding to a
regular latitude–longitude grid. Extensions to ESMValTool's regridding functions described here
allow for more flexible interpolation schemes that can be used on unstructured grids. Currently,
ESMValTool supports nearest-neighbor, bilinear, and first-order conservative regridding from
unstructured grids to regular grids.
Example applications of this new native model support are the evaluation of new model setups against
predecessor versions, assessing of the performance of different simulations against observations,
CMORization of native model data for contributions to model intercomparison projects, and monitoring
of running climate model simulations. For the latter, new general-purpose diagnostics have been added
to ESMValTool that are able to plot a wide range of variable types. Currently, five climate models
are supported: CESM2 (experimental; at the moment, only surface variables are available), EC-Earth3,
EMAC, ICON, and IPSL-CM6. As the framework for the CMOR-like reformatting of native model output
described here is implemented in a general way, support for other climate models can be easily added.
|
|
Weigel, K., Bock, L., Gier, B. K., Lauer, A., Righi, M., Schlund, M.,
Adeniyi, K., Andela, B., Arnone, E., Berg, P., Caron, L.-P., Cionni, I.,
Corti, S., Drost, N., Hunter, A., Lledó, L., Mohr, W. C., Paçal, A.,
Pérez-Zanón, N., Predoi, V., Sandstad, M., Sillmann, J., Sterl, A.,
Vegas-Regidor, J., von Hardenberg, J., and Eyring, V.: Earth System
Model Evaluation Tool (ESMValTool) v2.0 - diagnostics for extreme
events, regional and impact evaluation, and analysis of Earth system
models in CMIP, Geosci. Model Dev., 14, 3159-3184,
doi: 10.5194/gmd-14-3159-2021, 2021.
|
This paper complements a series of now four publications that
document the release of the Earth System Model Evaluation Tool
(ESMValTool) v2.0. It describes new diagnostics on the hydrological
cycle, extreme events, impact assessment, regional evaluations, and
ensemble member selection. The diagnostics are developed by a large
community of scientists aiming to facilitate the evaluation and
comparison of Earth system models (ESMs) which are participating in the
Coupled Model Intercomparison Project (CMIP). The second release of this
tool aims to support the evaluation of ESMs participating in CMIP
Phase 6 (CMIP6). Furthermore, datasets from other models and observations
can be analysed. The diagnostics for the hydrological cycle include several
precipitation and drought indices, as well as hydroclimatic intensity and
indices from the Expert Team on Climate Change Detection and Indices (ETCCDI).
The latter are also used for identification of extreme events, for impact
assessment, and to project and characterize the risks and impacts of
climate change for natural and socio-economic systems. Further impact
assessment diagnostics are included to compute daily temperature ranges
and capacity factors for wind and solar energy generation. Regional scales
can be analysed with new diagnostics implemented for selected regions and
stochastic downscaling. ESMValTool v2.0 also includes diagnostics to analyse
large multi-model ensembles including grouping and selecting ensemble members
by user-specified criteria. Here, we present examples for their capabilities
based on the well-established CMIP Phase 5 (CMIP5) dataset.
|