@@ -16,42 +16,55 @@ For detailed information, tutorials and more, please go to the
-## Showcase: Monitoring total seawater mass of an ACCESS CM2 run
+## Showcase: Monitoring total seawater mass of an ACCESS-CM2 run
-In our showcase, we will monitor the progress of an [ACCESS Coupled Model 2 (ACCESS-CM2)](/models/run-a-model/run-access-cm) run.
+In this showcase, monitoring the progress of an [ACCESS Coupled Model 2 (ACCESS-CM2)](/models/run-a-model/run-access-cm) run will be shown.
-We first start a session (for details on the paths and package see the documentation) to automatically check for new model output with a given period (here: 20 minutes):
+To start a session that automatically checks for new model output within a given period of 20 minutes:
```
import med_diagnostics
session = med_diagnostics.session.CreateModelDiagnosticsSession(model_type='CM2', model_path='path/to/your/live/model/data/output', period=20)
```
-Once a session is started, you will see the following sesion summary and blue status message while the new intake catalogue is being built from the live model data. Depending on the size of the model data, this can take a number of minutes.
+
+ For more details on paths and packages refer to the ACCESS-NRI Model Diagnostics documentation.
+
+
+When the session starts, you will see the following session summary:
+ The blue status message box appears while the new intake catalogue is being built from the live model data. Depending on the size of the model data, this can take several minutes.
+
+Once the live model data catalogue has been successfully built, the blue status message will update.
++The orange status message will report the time and date of the last live model catalogue build, as shown below:
+Select the dataset that you want to monitor (e.g., `ocean_scalar.1mon`) and click `Load dataset`.
+Use the dropdown menu to select and plot any available model variables listed.
diff --git a/docs/model_evaluation/model_evaluation_getting_started/model_evaluation_getting_started.md b/docs/model_evaluation/model_evaluation_getting_started/model_evaluation_getting_started.md
index e57a57212..5c183aeae 100644
--- a/docs/model_evaluation/model_evaluation_getting_started/model_evaluation_getting_started.md
+++ b/docs/model_evaluation/model_evaluation_getting_started/model_evaluation_getting_started.md
@@ -1,17 +1,17 @@
-# `conda` Environment for Model Evaluation on Gadi
+# Conda Environment for Model Evaluation on Gadi
-If you do not yet have `ssh` access to Gadi, refer to instructions on how to login to Gadi.
+If you do not have `ssh` access to Gadi, refer to instructions on how to login to Gadi.
-The following instructions explain how to load the curated `python` environment on NCI, which includes packages and scripts supported by ACCESS-NRI. Once loaded, these can be run directly on Gadi via `ssh`, `PBS` scripts, or in `JupyterLab`.
+The following instructions explain how to load the curated python environment on NCI, which includes packages and scripts supported by ACCESS-NRI. Once loaded, these can be run directly on Gadi via `ssh`, Portable Batch System (PBS) scripts, or in JupyterLab.
-???+ warning "ACCESS-NRI can provide code and support, but not computing resources"
+???+ warning "ACCESS-NRI provides code and support, but not computing resources"
You do not automatically have access to all `/g/data/` storage on Gadi. You need to join an NCI project to view files on `/g/data/$PROJECT`.
For model evaluation and diagnostics, you need to join projects `xp65` and `hh5` for code access and a `$PROJECT` with sufficient compute resources. ## What is the `access-med` environment? -The complete list of dependencies for the `access-med` environment can be found in the environment.yml file of the ACCESS-NRI MED GitHub repository. These include `intake`, `esmvaltool` and `ilamb`: +The complete list of dependencies for the `access-med` environment can be found in the environment.yml file of the ACCESS-NRI MED GitHub repository. Some of these include `intake`, `esmvaltool` and `ilamb`:
-To change to this curated environment, run the following commands after logging into Gadi and edit your `PBS` script accordingly: +To change to this curated environment, run the following commands after logging into Gadi and edit your PBS script accordingly: ``` module use /g/data/xp65/public/modules module load conda/access-med @@ -28,10 +28,15 @@ module load conda/access-med This will load the latest version of `access-med`, e.g. version `access-med-0.3`.
-To check which `conda` version you are using, run the following command: +To check which python version you are using, run the following command: ``` which python ``` +
-To submit this `PBS` job, execute the following command: +To submit your PBS job `example_pbs.sh`, run: ``` qsub example_pbs.sh ``` -In brief: this PBS script will submit a job to Gadi with the job name (`#PBS -N`) *example_pbs* under compute project (`#PBS -P`) `iq82` with a normal queue (`#PBS -q normalbw`), for 1 CPU (`#PBS -l ncpus=1`) with 2 GB RAM (`#PBS -l mem=2GB`), a walltime of 10 minutes (`#PBS -l walltime=00:10:00`) and data storage access to projects `xp65`. Note that for this example to work, you have to be member of the NCI project `xp65` and `iq82`. Adjust the `#PBS -P` option to match your compute project. Upon starting the job, it will change into to the working directory that you submitted the job from (`#PBS -l wd`) and load the access-med conda environment. - -This will submit a job to Gadi with the job name (`#PBS -N`) *example_pbs* under compute project (`#PBS -P`) *iq82* with a normalbw normal queue (`#PBS -q`). The number of CPUs requested is 1 CPU (`#PBS -l ncpus=1`) with 2 GB RAM (`#PBS -l mem=2GB`) and a walltime of 10 minutes (`#PBS -l walltime=00:10:00`). The data storage (`#PBS -l storage=gdata/xp65`) is data storage access to project `xp65`. +The above PBS script will submit a job to Gadi with the job name *example_pbs* (`#PBS -N`) under the `iq82` compute project (`#PBS -P`) in the `normalbw` queue (`#PBS -q`). It will use 1 CPU (`#PBS -l ncpus=1`), 2 GB RAM (`#PBS -l mem=2GB`), a walltime of 10 minutes (`#PBS -l walltime=00:10:00`) and data storage access to projects `xp65`.
Note: to run this example, you need to be a member of an NCI project, in this case `xp65` and `iq82` projects.
Adjust the `#PBS -P` option to match your compute project.
-When the job starts, it will change to the working directory from where you submitted the job (`#PBS -l wd`) and load the access-med `conda` environment. +When the job starts, it will change to the working directory from where you submitted the job (`#PBS -l wd`) and load the `access-med` conda environment.
-
-For more information on running `PBS` jobs on NCI, refer to PBS Jobs. + +For more information on running PBS jobs on Gadi, refer to PBS Jobs. ## Running the `access-med` environment on ARE -NCI also supports an interactive coding environment called Australian Research Environment (ARE). Its use is similar to submitting a `PBS` job via `qsub -I`, but with an added bonus of a dedicated graphical user interface for `Jupyter` notebooks. +NCI also supports an interactive coding environment called the Australian Research Environment (ARE). Its use is similar to submitting a PBS job via `qsub -I`, but with an added bonus of a dedicated graphical user interface for Jupyter notebooks.
-To use ARE, you must have an NCI account and be a member of a project with computing resources (see section on [getting started](../../getting_started/first_steps)). +For more information, check the Australian Research Environment (ARE) getting started. -Once you login to ARE, click on JupyterLab in the Featured Apps section to launch a `JupyterLab` instance. +Once you login to ARE, click on JupyterLab in the Featured Apps section to launch a JupyterLab instance.
Below are some example values that you should change to match your `$PROJECT` and use case: @@ -117,7 +120,7 @@ Below are some example values that you should change to match your `$PROJECT` an - **Modules** `conda/are` - *Launch* (click to submit) -This will launch a `JupyterLab` session with a Session ID, which will appear in the list of interactive sessions. (You can also find it under My Interactive Sessions at the top-left of the ARE window). +This will launch a JupyterLab session with a Session ID, which will appear in the list of interactive sessions. (You can also find it under My Interactive Sessions at the top-left of the ARE window).
The session appears blue while it is loading, yellow or red in case of warnings or errors, and green when it is successfully running: @@ -125,9 +128,9 @@ The session appears blue while it is loading, yellow or red in case of warnings
-This will open a window which contains a directory structure on the left and a `Jupyter` notebook on the right, as shown below. +This will open a window which contains a directory structure on the left and a Jupyter notebook on the right, as shown below.
If you loaded the modules from `hh5` or `xp65`, you should be able to import python packages such as `numpy`, `xarray` or `intake`, as shown below: diff --git a/docs/model_evaluation/model_evaluation_getting_started/model_variables/index.md b/docs/model_evaluation/model_evaluation_getting_started/model_variables/index.md index 5fab894af..75c03ec18 100644 --- a/docs/model_evaluation/model_evaluation_getting_started/model_variables/index.md +++ b/docs/model_evaluation/model_evaluation_getting_started/model_variables/index.md @@ -19,7 +19,7 @@ Numerous organisations and scientific groups worldwide have adopted a file forma
- Self-describing
-*.ncfiles include not only the data itself, but also a header with metadata that describes the data layout. +*.ncfiles include not only the data, but also a header with metadata that describes the data layout. - Machine-independent
@@ -27,7 +27,7 @@ Numerous organisations and scientific groups worldwide have adopted a file forma - Array-oriented
-*.ncdata typically spans multiple dimensions with the same lengths (e.g., latitude, longitude and time) and variables, such as temperature or humidity, which are stored in arrays. +*.ncdata typically spans multiple dimensions with the same lengths (e.g., latitude, longitude and time) and variables (e.g., temperature and humidity), which are stored in arrays.
@@ -37,13 +37,13 @@ Numerous organisations and scientific groups worldwide have adopted a file forma ### NetCDF metadata -Metadata, which is typically described as information about the data, enables users of data from different sources to decide which quantities are comparable. This facilitates building applications with powerful extraction, regridding and display capabilities. +Metadata, which is typically described as information about the data, enables users of data from different sources to decide which quantities are comparable. This facilitates building applications with powerful extraction, regridding and display capabilities. To facilitate this process, there are conventions for Climate and Forecast metadata. These are designed to promote the processing and sharing of NetCDF files. The conventions define metadata that provide a definitive description of what the data in each variable represents, and the spatial and temporal properties of the data. ### NetCDF data and variables -Data in a netCDF file is stored in the form of arrays, where each netCDF dimension has a name and a length. +Data in a NetCDF file is stored in the form of arrays, where each NetCDF dimension has a name and a length.+## Support + +To get help running your ESMValTool recipe on Gadi, you can submit an issue on the ESMValTool-Workflow GitHub repository or ask for help on the ACCESS-Hive Forum. + +General ESMValTool support (i.e. non-specific to Gadi) can be found on the ESMValTool Discussions page, where users can also post technical questions on the ESMValTool installation, application and development. + +### Recipes and diagnostics + +Contacts for specific diagnostic sets are the authors listed in the source code and corresponding recipe and diagnostic documentation. + +The current status of ESMValTool recipes for the `xp65` conda environment on Gadi is available here + +### License + +The ESMValTool is released under the Apache License, version 2.0. Citation of the ESMValTool paper (“Software Documentation Paper”) is requested upon use, along with the software DOI for ESMValTool (doi:10.5281/zenodo.3401363) and ESMValCore (doi:10.5281/zenodo.3387139) together with the version: + +> Righi, M., Andela, B., Eyring, V., Lauer, A., Predoi, V., Schlund, M., Vegas-Regidor, J., Bock, L., Brötz, B., de Mora, L., Diblen, F., Dreyer, L., Drost, N., Earnshaw, P., Hassler, B., Koldunov, N., Little, B., Loosveldt Tomas, S., and Zimmermann, K.: Earth System Model Evaluation Tool (ESMValTool) v2.0 – technical overview, Geosci. Model Dev., 13, 1179–1199, https://doi.org/10.5194/gmd-13-1179-2020, 2020. + +Besides the above citation, users are asked to register any journal articles (or other scientific documents) that use the software at the ESMValTool webpage (http://www.esmvaltool.org/). Citing the Software Documentation Paper and registering your paper(s) will serve to document the scientific impact of the Software, which is important for securing future funding. You should consider this an obligation if you have taken advantage of the ESMValTool, which represents the end product of considerable effort by the development team. + + - ILAMB/IOMB is a community-developed climate model diagnostics and evaluation software package. + ILAMB is a community-developed climate model diagnostics and evaluation software package. - ACCESS-NRI does not own the code of ILAMB/IOMB, but actively supports the use of ILAMB/IOMB on Gadi. - ILAMB development is primarily performed by the RUBISCO Science Focus Area and supported by the RGMA Activity of the EESSD division of the BER program in the United States Department of Energy’s Office of Science. - ACCESS-NRI provides access to the latest version of ILAMB/IOMB via the `xp65` access-med conda environment deployed on Gadi. + While ACCESS-NRI does not own the code, it actively supports the use of ILAMB software on Gadi. +
For example, temperature variation over time at a fixed location is stored as a one-dimensional array, whereas temperature over a region (i.e. varying location) at a fixed time is stored as a two-dimensional array. Thus, three-dimensional (3D) data would be temperature varying with time over a region, and four-dimensional (4D) data would be temperature varying with time over a region with varying altitude. @@ -70,7 +70,7 @@ Our Model Evaluation and Diagnostics tools are based on the reading and storing
For more information, refer to a quick overview of xarray and xarray tutorials. -`xarray` is a python package avaliable through the `conda` environment on NCI. +`xarray` is a python package avaliable through the conda environment on NCI.
Hence, you can either use it directly (as shown below) or through the dataset capabilities of the [ACCESS-NRI Model Intake Catalog Tool](../../model_evaluation_model_catalogs/index.md). diff --git a/docs/model_evaluation/model_evaluation_model_catalogs/index.md b/docs/model_evaluation/model_evaluation_model_catalogs/index.md index 69a7efd0e..2f77a8691 100644 --- a/docs/model_evaluation/model_evaluation_model_catalogs/index.md +++ b/docs/model_evaluation/model_evaluation_model_catalogs/index.md @@ -24,21 +24,21 @@ The ACCESS-NRI Intake catalog enables users to find products that satisfy their A simple use case of the ACCESS-NRI Intake catalog is a user wants to plot a timeseries of a variable from a specific data product.
-For example, the user is interested in plotting a scalar ocean variable called temp_global_ave for an [ACCESS-ESM1-5](/models/configurations/access-esm) run called `HI_CN_05` (data product), which is an historical run using the same configuration as CMIP6 ACCESS-ESM1.5 historical `r1i1p1f1`, but with phosphorus limitation disabled within CASA-CNP. +For example, the user is interested in plotting a scalar ocean variable called temp_global_ave for an [ACCESS-ESM1.5](/models/configurations/access-esm) run called HI_CN_05 (data product). This is an historical run using the same configuration as CMIP6 ACCESS-ESM1.5 historical r1i1p1f1, except that the phosphorus limitation within CASA-CNP is disabled. -First we load the catalog using +First, load the catalog as follows: ```python import intake catalog = intake.cat.access_nri ``` -To see all the available catalogs, simply prompt +To see all available catalogs, run: ``` catalog ``` -Now we can load and plot available datasets of the variable "temp_global_ave" from the product "HI_CN_05" using +Now you can load and plot available datasets for the temp_global_ave variable of the HI_CN_05 data product as follows: ```python import matplotlib.pyplot as plt diff --git a/docs/model_evaluation/model_evaluation_model_catalogs/model_evaluation_search_models.md b/docs/model_evaluation/model_evaluation_model_catalogs/model_evaluation_search_models.md index d8d0775e2..9029bd8bd 100644 --- a/docs/model_evaluation/model_evaluation_model_catalogs/model_evaluation_search_models.md +++ b/docs/model_evaluation/model_evaluation_model_catalogs/model_evaluation_search_models.md @@ -1,4 +1,4 @@ -# Search for a model in the ACCESS-NRI intake Catalog +# Search for a model in the ACCESS-NRI Intake catalog To have the huge amount of data from different experiments on the NCI storage at the palm of your hand, we provide a ("meta") catalog for you to query via python as part of the `#!python intake` package with our curated catalog plugin `#!python intake.cat.access_nri` . diff --git a/docs/model_evaluation/model_evaluation_on_gadi/index.md b/docs/model_evaluation/model_evaluation_on_gadi/index.md index 17f0a33a6..bce9227f8 100644 --- a/docs/model_evaluation/model_evaluation_on_gadi/index.md +++ b/docs/model_evaluation/model_evaluation_on_gadi/index.md @@ -1,7 +1,6 @@ -# Model Evaluation on Gadi/NCI - -At the moment, we are providing support for an the following model evaluation frameworks on Gadi/NCI: +# Model Evaluation on Gadi +ACCESS-NRI currently provides support for the following model evaluation frameworks on Gadi: -The best way to get our help is by raising an issue on the community forum with tags `help` and another tag for the specific framework. - -In the future, we are also aiming to support a broader range of frameworks and recipes which are currently not supported (see [our community resource lists](../../community_resources/community_med/index.md) for this collection). \ No newline at end of file + +
+In future, ACCESS-NRI aims to support a broader range of frameworks and recipes that are currently not supported, but are listed in Community Resources. \ No newline at end of file diff --git a/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_esmvaltool.md b/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_esmvaltool.md index 79c5a2918..76e15a240 100644 --- a/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_esmvaltool.md +++ b/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_esmvaltool.md @@ -1,13 +1,16 @@ -# Tutorial for using `esmvaltool` on Gadi@NCI +# Tutorial for using ESMValTool on Gadi -`esmvaltool` is the Earth System Model Evaluation Tool. +## What is ESMValTool? +The Earth System Model Evaluation Tool (ESMValTool) is a climate model diagnostics and evaluation software package to better understand the causes and effects of model biases and inter-model spread. -???+ warning "Support Level: Supported on Gadi, but not owned by ACCESS-NRI" - - ESMValTool is a community-developed climate model diagnostics and evaluation software package. +???+ warning "Support Level: Supported on Gadi, but not owned by ACCESS-NRI" - ACCESS-NRI does not own the code of ESMValTool, but actively supports the use of ESMValTool on Gadi. - ACCESS-NRI provides access to the latest version of ESMValTool via the `xp65` access-med conda environment deployed on NCI-Gadi. + ESMValTool is a community-developed climate model diagnostics and evaluation software package. While ACCESS-NRI does not own the code, it actively supports the use of ESMValTool software on Gadi. ACCESS-NRI provides access to the latest version of ESMValTool via the `xp65` `access-med` conda environment for Model Evaluation on Gadi. + + +The ESMValTool mainly focuses on evaluating results from the Coupled Model Intercomparison Project (CMIP) ensemble. + +For more information, refer to the official ESMValTool documentation. -## About ESMValTool - -The Earth System Model Evaluation Tool (ESMValTool) is a community-development that aims at improving diagnosing and understanding of the causes and effects of model biases and inter-model spread. The ESMValTool mainly focus on evaluating results from the Coupled Model Intercomparison Project (CMIP) ensemble. The goal is to build a common framework for the evaluation of Earth System Models (ESMs) against observations available through the Earth System Grid Federation (ESGF) in standard formats (obs4MIPs) or made available at ESGF nodes. - -More information on ESMValTool scope is available in the extensive ESMValTool documentation. - -ACCESS-NRI provides access to the latest version of ESMValTool via the `xp65` access-med conda environment deployed on NCI-Gadi. -Our plan is to routinely run benchmarks and comparisons of the ACCESS models CMIP submissions. We will also provide support for running recipes on NCI-Gadi. +## Using ESMValTool on Gadi -## Running `esmvaltool` on Gadi +ESMValTool is provided through the `xp65` `access-med` conda environment for Model Evaluation on Gadi. +
+ACCESS-NRI plans to routinely run benchmarks and comparisons of CMIP submissions for ACCESS models, as well as providing support to run ESMValTool recipes on Gadi. -#### Pre-requisites +### Pre-requisites -NCI Projects requires to run the set of ESMValTool recipes: +To run ESMValTool recipes, you need to be a member of the following NCI projects: -- xp65, ct11, fs38, oi10, rr3, al33, rt52, zz93, qv56 - - -#### Load the `access-med` conda environment +- `xp65`, `ct11`, `fs38`, `oi10`, `rr3`, `al33`, `rt52`, `zz93` and `qv56`. +### Running ESMValTool recipes + +To load the the `access-med` conda environment, execute the following commands: ``` module use /g/data/xp65/public/modules module load conda/access-med ``` -#### What recipes are available? +To list which ESMValTool recipes are available on Gadi, run: ``` esmvaltool recipes list ``` -#### Details of a recipe +To find out details of a specific `recipe_name.yml`, execute: ``` esmvaltool recipes show recipe_name.yml ``` -#### Running an recipe yourself +To execute `recipe_name.yml` and automatically download the required climate data to the default directory, run: ``` esmvaltool run examples/recipe_python.yml --search_esgf=when_missing ``` - -## Support - -ACCESS and NCI-Gadi users can get help from ACCESS-NRI for running their recipe on Gadi via Github Issue on the ESMValTool-Workflow github repository or by opening a thread on the ACCESS-Hive Forum. - -General support for ESMValTool (non-specific to NCI-Gadi) can be found in ESMValTool Discussions page where users can open an issue and a member of the User Engagement Team of ESMValTool will reply as soon as possible. This is open for all general and technical questions on the ESMValTool: installation, application, development, or any other question or comment you may have. - -### Recipes and diagnostics - -Contacts for specific diagnostic sets are the respective authors, as listed in the corresponding recipe and diagnostic documentation and in the source code. - -The current status of ESMValTool recipes for the `xp65` conda environment is available here - -## License - -The ESMValTool is released under the Apache License, version 2.0. Citation of the ESMValTool paper (“Software Documentation Paper”) is kindly requested upon use, alongside with the software DOI for ESMValTool (doi:10.5281/zenodo.3401363) and ESMValCore (doi:10.5281/zenodo.3387139) and version number: - -> Righi, M., Andela, B., Eyring, V., Lauer, A., Predoi, V., Schlund, M., Vegas-Regidor, J., Bock, L., Brötz, B., de Mora, L., Diblen, F., Dreyer, L., Drost, N., Earnshaw, P., Hassler, B., Koldunov, N., Little, B., Loosveldt Tomas, S., and Zimmermann, K.: Earth System Model Evaluation Tool (ESMValTool) v2.0 – technical overview, Geosci. Model Dev., 13, 1179–1199, https://doi.org/10.5194/gmd-13-1179-2020, 2020. - -Besides the above citation, users are kindly asked to register any journal articles (or other scientific documents) that use the software at the ESMValTool webpage (http://www.esmvaltool.org/). Citing the Software Documentation Paper and registering your paper(s) will serve to document the scientific impact of the Software, which is of vital importance for securing future funding. You should consider this an obligation if you have taken advantage of the ESMValTool, which represents the end product of considerable effort by the development team. +The `--search_esgf=when_missing` option tells ESMValTool to search for and download the necessary climate data files from Earth System Grid Federation (ESGF), if they cannot be found locally. ## ESMValTool recipe examples -To find the available recipes, please go see the ACCESS ESMValTool Worflow recipe status +A list of ESMValTool recipes available on Gadi can be found on the ACCESS-NRI MED ESMValTool Workflow GitHub repository. Some example recipes are provided below: -Below we showcase example recipes from `esmvaltool` that we are providing to run on Gadi: @@ -110,6 +88,27 @@ Below we showcase example recipes from `esmvaltool` that we are providing to run
+ + ACCESS-NRI provides access to the latest version of ILAMB via the `xp65` `access-med` conda environment for Model Evaluation on Gadi. + +This documentation is tailored to using ILAMB on Gadi and, hence, it supplements rather than replaces the official documentation. Users are encouraged to read the ILAMB documentation and relevant tutorials. -This documentation is tailored to using the tool within the NCI infrastructure and designed to supplement, rather than replace, the official documentation. We encourage users to read the ILAMB documentation and its Tutorials. +## Using ILAMB on Gadi -## Using ILAMB and IOMB on Gadi +ILAMB is provided through both the `xp65` and `hh5` NCI projects on Gadi, so you need to have an NCI account and be a member of these projects to use it. -`ilamb` is provided through both the `xp65` and `hh5` code projects on Gadi at the National Computational Infrastructure. To use `ilamb` through this infrastructure, you need to have an NCI account and join the projects. See our [Getting Started Guide](../../getting_started/first_steps.md). ++ To obtain an NCI account and join NCI projects, refer to First Steps. +-You will find the information needed to run ILAMB and IOMB on Gadi in our documentation: +For ACCESS-NRI documentation on how to run ILAMB on Gadi, visit:@@ -29,40 +34,52 @@ You will find the information needed to run ILAMB and IOMB on Gadi in our docume-To run `ilamb`, you need to execute the command `ilamb-run` with a number of arguments/files: +
+To run ILAMB, you need to execute the command `ilamb-run` with a number of arguments/ files: ``` ilamb-run --config config.cfg --model_setup model_setup.txt --regions regions.txt ``` +-
+
-
+
config.cfgdefines which observables and observational datasets to be compared. + -
+
model_setup.txtdefines the paths of models that to be compared. + -
+
regions.txtdefines the regions to be compared, e.g.,global,aust(Australia), etc. +
+NCI hosts replicas of the ILAMB observational data sets through the NCI project `ct11` as well as a large amount of model outputs are available on Gadi, such as ACCESS model output. -If you want to learn how to adjust the `ilamb` setup, you can also read the official ILAMB documentation and its Tutorial. +For more information, refer to Observational Data Catalogue on how to find observational data on NCI and ACCESS-NRI Intake Catalog for how to find model outputs. +
+To learn more about how to adjust the ILAMB setup, refer to the official ILAMB documentation and relevant tutorials. -## Showcases: CMIP6 comparisons and ACCESS ESM1.5 benchmarking +## Showcase: CMIP6 comparisons and ACCESS ESM1.5 benchmarking -ACCESS-NRI is maintaining a collection of benchmark comparisons for the community, such as the comparison of Coupled Model Intercomparison Project (CMIP) like +ACCESS-NRI is maintaining a collection of benchmark comparisons for the ACCESS community, such as that with Coupled Model Intercomparison Project (CMIP) data: -- CMIP5 and CMIP6 Land Models, -- CMIP6 Land Models, -- Offline CMIP6 Models, and -- CMIP5 and CMIP6 Ocean Models. +- CMIP5 and CMIP6 Land Models +- CMIP6 Land Models +- Offline CMIP6 Models +- CMIP5 and CMIP6 Ocean Models -For our showcase, however, we are comparing the ACCESS Earth System Model version 1.5 that is supported by ACCESS-NRI with two other Earth System Models: +In the following example, the supported ACCESS Earth System Model (ESM) ACCESS-ESM1.5 is compared with two other ESM models: -- ACCESS-ESM1.5 (ACCESS Earth System Model version 1.5), -- BCC ESM1 (Beijing Climate Center Earth System Model version 1), and +- BCC ESM1 (Beijing Climate Center Earth System Model version 1) - CanESM5 (Canadian Earth System Model version 5) -We have performed a large amount of benchmark comparisons that were defined in the configuration file. We have organised the comparison of variables under different sections, like the Hydrology Cycle. For different variables, like the gross primary productivity `gpp`, we can compare to one or more datasets, like the gross primary productivity measurements of FLUXNET2015. Clicking on a row of the table will expand it to reveal the underlying datasets used. In the below table, the colormap extends from best values in purple to worse data in orange. +Numerous benchmark comparisons have been defined in the configuration file. The comparison of variables have been organised under different sections, such as the Hydrology Cycle. +
+For other variables, such as the git Gross Primary Productivity `gpp`, one or more datasets are available. For example, the gross primary productivity measurements of FLUXNET2015. +
+
+By clicking on a row in the table, you can expand it to see the underlying datasets used. The table's colourmap extends from best values in purple to worse data in orange.
-Clicking on one of these datasets, for example CERESed4.1, will take you to an interactive and quantitative comparison page for Albedo measurements of the Clouds and the Earth’s Radiant Energy System (CERES) project: +
+Clicking on one of these datasets, for example `CERESed4.1`, will take you to an interactive and quantitative comparison page for Albedo measurements of the Clouds and the Earth’s Radiant Energy System (CERES) project:
\ No newline at end of file diff --git a/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_metplus.md b/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_metplus.md index f5cf8e41f..82b1ebf50 100644 --- a/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_metplus.md +++ b/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_metplus.md @@ -1,15 +1,18 @@ -# `METplus` on Gadi at NCI +# METplus on Gadi -METplus is the enhanced Model Evaluation Tools (METplus) verification system. +METplus is the enhanced Model Evaluation Tools verification framework that spans a wide range of temporal (warn-on-forecast to climate) and spatial (storm to global) scales. +
+
+The core components of the framework include the Model Evaluation Tools (MET), the associated database and display systems (METviewer and METexpress), and a suite of Python wrappers to provide low-level automation and examples. -???+ warning "Support Level: Supported on Gadi, but not owned by ACCESS-NRI" +???+ warning "Support Level: Supported on Gadi, but not owned by ACCESS-NRI" - METplus was developed by the Developmental Testbed Center (DTC) and is being actively developed by NCAR/Research Applications Laboratory (RAL), NOAA/Earth Systems Research Laboratories (ESRL), NOAA/Environmental Modeling Center (EMC), and is open to community contributions. + - ACCESS-NRI does not own the code of METplus, but actively supports the use of METplus on Gadi. - ACCESS-NRI provides access to the latest version of ESMValTool via the `access` conda environment deployed on NCI-Gadi. + While ACCESS-NRI does not own the code, it actively supports the use of the METplus on Gadi. + ACCESS-NRI provides access to the latest version of METplus via the `access` conda environment for Model Evaluation on Gadi. -For detailed information, tutorials and more of METplus, please go to the +For more information and tutorials, refer to the official METplus documentation:@@ -19,32 +22,36 @@ For detailed information, tutorials and more of METplus is a verification framework that spans a wide range of temporal (warn-on-forecast to climate) and spatial (storm to global) scales. It is intended to be extensible through additional capability developed by the community The core components of the framework include the Model Evaluation Tools (MET), the associated database and display systems called METviewer and METexpress, and a suite of Python wrappers to provide low-level automation and examples, also called use-cases. METplus will be a component of NOAA's Unified Forecast System (UFS) cross-cutting infrastructure as well as NCAR's System for Integrated Modeling of the Atmosphere (SIMA). +## Using METplus on Gadi -## Showcase of METplus 5.0 - -To load all METplus, and have all commands and run_metplus.py available through `$PATH` on Gadi, load the METplus module as part of NCI project `access`: +Load the `metplus` module as part of NCI project `access` to have all the commands and `run_metplus.py` through `$PATH` on Gadi, run the following: ``` module use /g/data/access/ngm/modules module load envs/metplus/5.0 ``` - -1. Download the sample data from https://dtcenter.ucar.edu/dfiles/code/METplus/METplus_Data/v5.0/sample_data-met_tool_wrapper-5.0.tgz and untar into a directory on Gadi, for example `~/METplus`. - -2. Create a configuration file `local.conf` containing the input and output paths, for example `INPUT_BASE=~/METplus`. - +
+Then run through the following steps: +
+
+1. Sample data can be downloaded from https://dtcenter.ucar.edu/dfiles/code/METplus/METplus_Data/v5.0/sample_data-met_tool_wrapper-5.0.tgz +
+You then need to `untar` the data in a directory on Gadi. +
+For example, `~/METplus` directory. +
+
+2. Create a configuration file `local.conf` containing the input and output paths, where for example `INPUT_BASE=~/METplus`: ``` [dir] INPUT_BASE=/path/to/metplus_inputs OUTPUT_BASE=/path/to/outputs ``` - -3. Save the demo configuration (e.g. `ASCII2NC.conf` from this METPlus example to a local file - -4. Run METplus passing it both local.conf and the demo configuration - +
+3. Save the demo configuration (e.g., `ASCII2NC.conf`) from this METPlus example to a local file. +
+
+4. Run METplus, passing it both the `local.conf` file and demo configuration, as follows: ``` run_metplus.py local.conf ASCII2NC.conf ``` diff --git a/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_pangeo_cosima.md b/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_pangeo_cosima.md index 7ea31adbb..fc30eac97 100644 --- a/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_pangeo_cosima.md +++ b/docs/model_evaluation/model_evaluation_on_gadi/model_evaluation_on_gadi_pangeo_cosima.md @@ -1,37 +1,41 @@ -# COSIMA Cookbook on NCI's Gadi +# COSIMA Cookbook on Gadi -???+ warning "Support Level: Supported on Gadi, but not owned by ACCESS-NRI" - - The COSIMA Cookbook is developed and maintained by the Consortium for Ocean-Sea Ice Modelling in Australia. - - ACCESS-NRI does not own the code of the COSIMA Cookbook, but actively supports the use of the COSIMA Cookbook and its collection of `cosima-recipes` on Gadi. - ACCESS-NRI provides access to the latest version of the COSIMA Cookbook via the `hh5` access-med conda environment deployed on NCI-Gadi. -COSIMA is the Consortium for Ocean-Sea Ice Modelling in Australia, which brings together Australian researchers involved in global ocean and sea ice modelling. The consortium provides a collection of `cosima-recipes` for the evaluation of ocean-sea ice modelling that are currated for you on Gadi. +COSIMA is the Consortium for Ocean-Sea Ice Modelling in Australia, which brings together Australian researchers involved in global ocean and sea ice modelling. The COSIMA Cookbook is a collection of `cosima-recipes` curated on Gadi for analysing output from ocean-sea ice models. + +???+ warning "Support Level: Supported on Gadi, but not owned by ACCESS-NRI" + + The COSIMA Cookbook is developed and maintained by COSIMA. While ACCESS-NRI does not own the code, it actively supports the use of the COSIMA Cookbook and its collection of `cosima-recipes` on Gadi. + ACCESS-NRI provides access to the latest version of COSIMA Cookbook via the `hh5` `access-med` conda environment for Model Evaluation on Gadi. -The COSIMA Cookbook is a framework for analysing output from ocean-sea ice models. The focus is on the [ACCESS-OM2](../../models/configurations/access-om.md) suite of models being developed and run by members of COSIMA. But this framework is suited to analysing any MOM5/MOM6 output, as well as output from other models. + The COSIMA Cookbook framework focuses on the [ACCESS-OM2](../../models/configurations/access-om.md) suite of models being developed and run by members of COSIMA. Nevertheless, this framework is suited to analysing any MOM5/ MOM6 output as well as output from other models. ## Getting Started -The easiest way to use the COSIMA Cookbook is through the Australian Research Environment (ARE) access of the National Computational Infrastructure. Here, we assume that you already [got started](/getting_started/first_steps), that is, you have an NCI account and can log onto Gadi via secure shell (ssh). +The easiest way to use the COSIMA Cookbook is through the Australian Research Environment (ARE) on Gadi.
+You need to have an NCI account and be able to `ssh` login to Gadi (see section on [first steps](../../getting_started/first_steps.md)). + +To use the COSIMA Cookbook that is pre-installed in the `conda/analysis3` environment of `hh5`, you need to join NCI project `hh5`. + +1. Login via `ssh` to Gadi and clone thecosima-recipesrepository to your local directory. -To use the COSIMA Cookbook that is preinstalled in the `conda/analysis3` of NCI proejct `hh5`, you need to join NCI project `hh5`. +2. Find the recipes that you want to run and make sure you have access to the specific projects and their storage (e.g., project `ik11` to get access to `/g/data/ik11`). -1. Log onto Gadi via secure shell (ssh) and clone thecosima-recipesrepository to your local file space. -2. Check out the recipes that you want to run, and make sure that you have access to the specific projects and their storage (e.g. project `ik11` to get access to `/g/data/ik11`). -3. Start an ARE JupyterLab session on NCI: - **Storage**: gdata/hh5 (add the specific storage that you need for the recipes you want to run) - **Module directories**: /g/data/hh5/public/modules - **Modules**: conda/analysis3 - You can check out our [Getting Started with ARE](../model_evaluation_getting_started/model_evaluation_getting_started.md) instructions if you have not used ARE before. -1. Within the ARE environment, navigate to one of the COSIMA recipes and run the analysis. +3. Start an ARE JupyterLab session on NCI: + Storage: `gdata/hh5` (add the specific storage you need for the recipe you want to run) +
+ Module directories: `/g/data/hh5/public/modules` + Modules: `conda/analysis3` +
+ If you are new to ARE, refer to instructions on [Getting Started with ARE](../model_evaluation_getting_started/model_evaluation_getting_started.md). +4. Within the ARE environment, navigate to one of the COSIMA recipes and run the analysis. -## More information about the Cookbook +## COSIMA Cookbook information -For more information, we refer to the Cookbook github repository as well as a list of recipes: +For more information on the COSIMA Cookbook, refer to the cosima-cookbook GitHub repository, as well as the following lists of recipes: -- Tutorials, -- Notebooks to reproduce figures of the ACCESS-OM2 announcement paper, -- Documented Example, and +- Tutorials +- Notebooks to reproduce figures of the ACCESS-OM2 announcement paper +- Documented Example - Contributed Examples -
+