Skip to content

Commit

Permalink
Merge pull request #568 from ACCESS-Hive/dev/heidi/issue560
Browse files Browse the repository at this point in the history
Edited Data and Model Evaluation section
  • Loading branch information
atteggiani authored Sep 3, 2023
2 parents a652b75 + c2ce8a0 commit 1e5eacf
Show file tree
Hide file tree
Showing 11 changed files with 218 additions and 175 deletions.
3 changes: 2 additions & 1 deletion docs/getting_started/are.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@
<br>
ARE can give you access to NCI’s <i>Gadi</i> supercomputer and data collections.


There are multiple applications included in ARE, but the two most used for ACCESS-related activities are <a href="#vdi">Virtual Desktop (VDI)</a> and <a href="#jupyterlab">JupyterLab</a>.

## Prerequisites
To use ARE, you must have an NCI account and be a member of a project with computing resources (SU). If you are new to ACCESS, follow the <a href="/getting_started/first_steps">First Steps</a>.
## Start an ARE session
<!-- Tab labels -->
<div class="tabLabels" label="are-apps" style="margin-bottom: 0.5rem;">
Expand Down
35 changes: 24 additions & 11 deletions docs/model_evaluation/model_diagnostics/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

## What is Model Live Diagnostics?

The Model Live Diagnostics framework is a simple, easy to use and accessible Jupter-based framework for the ACCESS modelling community to check, monitor, visualise and evaluate model behaviour and progress on currently running or ‘live’ ACCESS models on the Australian NCI supercomputer Gadi.
<i>Model Live Diagnostics</i> is a simple, accessible and easy to use Jupter-based framework for the ACCESS modelling community to monitor, visualise and evaluate the behaviour of models in real time (<i>live</i>) while they run on <i>Gadi</i>.

In addition to monitoring a live model, the package provides the functionality to load, visualise and compare legacy ACCESS model data with the selected live user model.
In addition to monitoring a live model, the package also provides the functionality to load, visualise and compare legacy ACCESS model data with the live model.

For detailed information, tutorials and more, please go to the
For more information and tutorials, please visit:
<div class="card-container">
<a href="https://med-live-diagnostics.readthedocs.io/en/latest/index.html" class="vertical-card aspect-ratio2to1" target="_blank">
<div class="card-image-container">
Expand All @@ -16,42 +16,55 @@ For detailed information, tutorials and more, please go to the
</a>
</div>

## Showcase: Monitoring total seawater mass of an ACCESS CM2 run
## Showcase: Monitoring total seawater mass of an ACCESS-CM2 run

In our showcase, we will monitor the progress of an [ACCESS Coupled Model 2 (ACCESS-CM2)](/models/run-a-model/run-access-cm) run.
In this showcase, monitoring the progress of an [ACCESS Coupled Model 2 (ACCESS-CM2)](/models/run-a-model/run-access-cm) run will be shown.

We first start a session (for details on the paths and package see the <a href="https://med-live-diagnostics.readthedocs.io/en/latest/index.html" target="_blank">documentation</a>) to automatically check for new model output with a given period (here: 20 minutes):
To start a session that automatically checks for new model output within a given period of 20 minutes:

```
import med_diagnostics
session = med_diagnostics.session.CreateModelDiagnosticsSession(model_type='CM2', model_path='path/to/your/live/model/data/output', period=20)
```

Once a session is started, you will see the following sesion summary and blue status message while the new intake catalogue is being built from the live model data. Depending on the size of the model data, this can take a number of minutes.
<div class="note">
For more details on paths and packages refer to the <a href="https://med-live-diagnostics.readthedocs.io/en/latest/index.html" target="_blank">ACCESS-NRI Model Diagnostics documentation</a>.
</div>

When the session starts, you will see the following session summary:

<div style="text-align: center;">
<img src="../../assets/model_evaluation/live_diagnostics/tutorial_image_1.png" alt="Output of the Model Live Diagnostics after a session has been started and a new catalogue is being built." width="75%"/>
</div>

Once the live model data catalogue has been successfully built, the blue status message will update and the orange status message will report the time and date of the last live model catalogue build.
<div class="note">
The blue status message box appears while the new intake catalogue is being built from the live model data. Depending on the size of the model data, this can take several minutes.
</div>

Once the live model data catalogue has been successfully built, the blue status message will update.
<br>
The orange status message will report the time and date of the last live model catalogue build, as shown below:
<div style="text-align: center;">
<img src="../../assets/model_evaluation/live_diagnostics/tutorial_image_2.png" alt="Output of the Model Live Diagnostics after the catalogue has been built." width="75%"/>
</div>

All available datasets from the selected model will be listed in the dropdown. Select the dataset you wish to monitor and click ‘Load dataset’.
All available datasets for the selected model will be listed in the dropdown menu.
<br>
Select the dataset that you want to monitor (e.g., `ocean_scalar.1mon`) and click `Load dataset`.

<div style="text-align: center;">
<img src="../../assets/model_evaluation/live_diagnostics/tutorial_image_3.png" alt="Output of the Model Live Diagnostics with a dropdown menu of available datasets." width="75%"/>
</div>

Once loaded, a plot displaying the first data variable in the list will appear. Use the dropdown list to select and plot any available model variables.
Once loaded, a plot will display the first data variable selected from the list.
<br>
Use the dropdown menu to select and plot any available model variables listed.

<div style="text-align: center;">
<img src="../../assets/model_evaluation/live_diagnostics/tutorial_image_4.png" alt="Plot of total liquid seawater mass over time of the ‘live’ ACCES CM2 run." width="75%"/>
</div>

With a few more clicks, you can also load legacy data to compare with, for example other CM2 models like by578 or by578a:
It is also possible to load and compare <i>legacy data</i>, such as other ACCESS-CM2 models <i>by578</i> and <i>by578a</i>:

<div style="text-align: center;">
<img src="../../assets/model_evaluation/live_diagnostics/tutorial_image_7.png" alt="Plot of total liquid seawater mass over time of the ‘live’ ACCES CM2 run when compared to legacy model data." width="75%"/>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# `conda` Environment for Model Evaluation on Gadi
# Conda Environment for Model Evaluation on Gadi

If you do not yet have `ssh` access to <i>Gadi</i>, refer to instructions on how to <a href="/getting_started/first_steps#login-to-gadi">login to Gadi</a>.
If you do not have `ssh` access to <i>Gadi</i>, refer to instructions on how to <a href="/getting_started/first_steps#login-to-gadi">login to Gadi</a>.

The following instructions explain how to load the curated `python` environment on NCI, which includes packages and scripts supported by ACCESS-NRI. Once loaded, these can be run directly on <i>Gadi</i> via `ssh`, `PBS` scripts, or in `JupyterLab`.
The following instructions explain how to load the curated python environment on NCI, which includes packages and scripts supported by ACCESS-NRI. Once loaded, these can be run directly on <i>Gadi</i> via `ssh`, Portable Batch System (PBS) scripts, or in JupyterLab.

???+ warning "ACCESS-NRI can provide code and support, but not computing resources"
???+ warning "ACCESS-NRI provides code and support, but not computing resources"
You do not automatically have access to all `/g/data/` storage on <i>Gadi</i>. You need to <a href="/getting_started/first_steps#join-relevant-nci-projects">join an NCI project</a> to view files on `/g/data/$PROJECT`.
<br>
For model evaluation and diagnostics, you need to join projects `xp65` and `hh5` for code access and a `$PROJECT` with sufficient compute resources.

## What is the `access-med` environment?

The complete list of dependencies for the `access-med` environment can be found in the <a href="https://github.com/ACCESS-NRI/MED-condaenv/blob/main/scripts/environment.yml" target="_blank">environment.yml</a> file of the <a href="https://github.com/ACCESS-NRI/MED-condaenv" target="_blank">ACCESS-NRI MED GitHub repository</a>. These include `intake`, `esmvaltool` and `ilamb`:
The complete list of dependencies for the `access-med` environment can be found in the <a href="https://github.com/ACCESS-NRI/MED-condaenv/blob/main/scripts/environment.yml" target="_blank">environment.yml</a> file of the <a href="https://github.com/ACCESS-NRI/MED-condaenv" target="_blank">ACCESS-NRI MED GitHub repository</a>. Some of these include `intake`, `esmvaltool` and `ilamb`:
<div style="text-align: center;">
<img src="../../../assets/model_evaluation/condaenv_list.png" alt="List of packages that are provided as part of the xp65 access-med environment" width="75%"/>
</div>
Expand All @@ -20,18 +20,23 @@ The complete list of dependencies for the `access-med` environment can be found

To avoid running code on <i>Gadi</i> with incompatible packages, a conda environment called `access-med` is provided.
<br>
To change to this curated environment, run the following commands after logging into <i>Gadi</i> and <a href="https://opus.nci.org.au/display/Help/PBS+Directives+Explained" target="_blank">edit</a> your `PBS` script accordingly:
To change to this curated environment, run the following commands after logging into <i>Gadi</i> and <a href="https://opus.nci.org.au/display/Help/PBS+Directives+Explained" target="_blank">edit your PBS script</a> accordingly:
```
module use /g/data/xp65/public/modules
module load conda/access-med
```

This will load the latest version of `access-med`, e.g. version `access-med-0.3`.
<br>
To check which `conda` version you are using, run the following command:
To check which python version you are using, run the following command:
```
which python
```
<!-- For details about the conda environment you are using, run the following command:
```
conda info --envs
```
-->

<terminal-window>
<terminal-line data="input">module use /g/data/xp65/public/modules</terminal-line>
Expand All @@ -43,7 +48,7 @@ which python
<terminal-line>/g/data/xp65/public/apps/med_conda_scripts/access-med-0.3.d/bin/python</terminal-line>
</terminal-window>

To test everything is working correctly, import the packages in `python3` as follows:
To test everything is working correctly, import the packages in python3 as follows:

```python
import numpy as np
Expand All @@ -56,7 +61,7 @@ print(intake.__version__)
print(esmvaltool.__version__)
```

If you want to run your code on <i>Gadi</i> using a Portable Batch System (`PBS`) job, add the `module use` and `module load` commands to your `PBS` script as shown in the `example_pbs.sh` `PBS` script below:
If you want to run your code on <i>Gadi</i> using a PBS job, add the `module use` and `module load` commands to your PBS script as shown in the `example_pbs.sh` PBS script below:

```
#!/bin/bash
Expand All @@ -75,35 +80,33 @@ module load conda/access-med
python3 your_code.py
```

The content of `your_code.py` could simply comprise the `import` and `which version` lines from our above example.
The content of `your_code.py` could simply comprise a few lines, such as which conda version and which packages to import.
<br>
To submit this `PBS` job, execute the following command:
To submit your PBS job `example_pbs.sh`, run:
```
qsub example_pbs.sh
```

In brief: this PBS script will submit a job to Gadi with the job name (`#PBS -N`) *example_pbs* under compute project (`#PBS -P`) `iq82` with a <a href="https://opus.nci.org.au/display/Help/Queue+Limits" target="_blank">normal queue</a> (`#PBS -q normalbw`), for 1 CPU (`#PBS -l ncpus=1`) with 2 GB RAM (`#PBS -l mem=2GB`), a walltime of 10 minutes (`#PBS -l walltime=00:10:00`) and data storage access to projects `xp65`. Note that for this example to work, you have to be <a href="https://my.nci.org.au/mancini/project-search" target="_blank">member of the NCI project</a> `xp65` and `iq82`. Adjust the `#PBS -P` option to match your compute project. Upon starting the job, it will change into to the working directory that you submitted the job from (`#PBS -l wd`) and load the access-med conda environment.

This will submit a job to <i>Gadi</i> with the job name (`#PBS -N`) *example_pbs* under compute project (`#PBS -P`) *iq82* with a </i>normalbw</i> <a href="https://opus.nci.org.au/display/Help/Queue+Limits" target="_blank">normal queue</a> (`#PBS -q`). The </i>number of CPUs</i> requested is 1 CPU (`#PBS -l ncpus=1`) with 2 GB RAM (`#PBS -l mem=2GB`) and a <i>walltime</i> of 10 minutes (`#PBS -l walltime=00:10:00`). The <i>data storage</i> (`#PBS -l storage=gdata/xp65`) is data storage access to project `xp65`.
The above PBS script will submit a job to <i>Gadi</i> with the job name *example_pbs* (`#PBS -N`) under the `iq82` compute project (`#PBS -P`) in the <a href="https://opus.nci.org.au/display/Help/Queue+Limits" target="_blank">`normalbw` queue</a> (`#PBS -q`). It will use 1 CPU (`#PBS -l ncpus=1`), 2 GB RAM (`#PBS -l mem=2GB`), a walltime of 10 minutes (`#PBS -l walltime=00:10:00`) and data storage access to projects `xp65`.
<br>
<br>
<i>Note</i>: to run this example, you need to be a <a href="https://my.nci.org.au/mancini/project-search" target="_blank">member of an NCI project</a>, in this case `xp65` and `iq82` projects.
<br>
Adjust the `#PBS -P` option to match your compute project.
<br>
When the job starts, it will change to the working directory from where you submitted the job (`#PBS -l wd`) and load the access-med `conda` environment.
When the job starts, it will change to the working directory from where you submitted the job (`#PBS -l wd`) and load the `access-med` conda environment.
<br>
<br>
For more information on running `PBS` jobs on NCI, refer to <a href="https://opus.nci.org.au/display/Help/4.+PBS+Jobs" target="_blank">PBS Jobs</a>.

For more information on running PBS jobs on <i>Gadi</i>, refer to <a href="https://opus.nci.org.au/display/Help/4.+PBS+Jobs" target="_blank">PBS Jobs</a>.

## Running the `access-med` environment on ARE

NCI also supports an interactive coding environment called Australian Research Environment (<i>ARE</i>). Its use is similar to submitting a `PBS` job via `qsub -I`, but with an added bonus of a dedicated graphical user interface for `Jupyter` notebooks.
NCI also supports an interactive coding environment called the Australian Research Environment (ARE). Its use is similar to submitting a PBS job via `qsub -I`, but with an added bonus of a dedicated graphical user interface for Jupyter notebooks.
<br>
<br>
To use <i>ARE</i>, you must have an NCI account and be a member of a project with computing resources (see section on [getting started](../../getting_started/first_steps)).
For more information, check the <a href="/getting_started/are">Australian Research Environment (ARE) getting started</a>.

Once you <a href="https://are.nci.org.au" target="_blank">login to <i>ARE</i></a>, click on <i>JupyterLab</i> in the <i>Featured Apps</i> section to launch a `JupyterLab` instance.
Once you <a href="https://are.nci.org.au" target="_blank">login to <i>ARE</i></a>, click on <i>JupyterLab</i> in the <i>Featured Apps</i> section to launch a JupyterLab instance.
<br>
Below are some example values that you should change to match your `$PROJECT` and use case:

Expand All @@ -117,17 +120,17 @@ Below are some example values that you should change to match your `$PROJECT` an
- **Modules** `conda/are`
- *Launch* (click to submit)

This will launch a `JupyterLab` session with a <i>Session ID</i>, which will appear in the list of interactive sessions. (You can also find it under <i>My Interactive Sessions</i> at the top-left of the ARE window).
This will launch a JupyterLab session with a <i>Session ID</i>, which will appear in the list of interactive sessions. (You can also find it under <i>My Interactive Sessions</i> at the top-left of the ARE window).
<br>
The session appears blue while it is loading, yellow or red in case of warnings or errors, and green when it is successfully running:

<div style="text-align: center;">
<img src="../../../assets/getting_started/are_1.png" alt="Example of a successfully started ARE Session" width="75%"/>
</div>

You can then <i>Open JupyterLab</i> by clicking on the button at the bottom of the session.
Launch JupyterLab by clicking on the <i>Open JupyterLab</i> button at the bottom of the session.
<br>
This will open a window which contains a directory structure on the left and a `Jupyter` notebook on the right, as shown below.
This will open a window which contains a directory structure on the left and a Jupyter notebook on the right, as shown below.
<br>
If you loaded the modules from `hh5` or `xp65`, you should be able to import python packages such as `numpy`, `xarray` or `intake`, as shown below:

Expand Down
Loading

0 comments on commit 1e5eacf

Please sign in to comment.