Merge pull request #141 from rosswhitfield/docs_updates

Update docs about cori tests, _etc_. Bump static analysis versions

Author: rosswhitfield

.github/workflows/workflows.yml

@@ -16,7 +16,7 @@ jobs:
       with:
         python-version: 3.8
     - name: Install python testing dependencies
-      run: pip install flake8==3.9.2 pylint==2.9.5 bandit==1.7.0 codespell==2.1.0 dask distributed
+      run: pip install flake8==3.9.2 pylint==2.9.6 bandit==1.7.0 codespell==2.1.0 dask distributed
     - name: flake8
       run: flake8
     - name: pylint
@@ -36,7 +36,7 @@ jobs:
       matrix:
         os: [ubuntu-latest, macos-latest]
         python-version: [3.6, 3.7, 3.8, 3.9]
-        dask-version: ['2021.07.1']
+        dask-version: ['2021.07.2']
         include:
           # same as NERSC Cori module python/3.7-anaconda-2019.10
           - os: ubuntu-latest
@@ -97,7 +97,7 @@ jobs:
               python3-pip
               python-is-python3
     - name: Install python testing dependencies
-      run: python -m pip install pytest-cov pytest-timeout psutil dask==2021.07.1 distributed==2021.07.1
+      run: python -m pip install pytest-cov pytest-timeout psutil dask==2021.07.2 distributed==2021.07.2
     - name: Install IPS (in develop mode)
       run: python -m pip install -e .
     - name: testing running IPS (--help)

doc/development.rst

@@ -135,6 +135,37 @@ You can then also run ``python -m coverage report -m`` to show exactly
 which lines are missing test coverage.
 
 
+.. _cori-tests :
+
+Cori only tests
+~~~~~~~~~~~~~~~
+
+The are some tests that only run on Cori at NERSC and these are not
+run as part of the :ref:`CI <continuous integration>` and must be run
+manually. To run those test you need to add the option ``--runcori``
+to the ``pytest``. There are tests for the :ref:`shifter
+functionally<dask_shifter>` that is Cori specific.
+
+An example batch script for running the unit tests is:
+
+.. code-block:: bash
+
+  #!/bin/bash
+  #SBATCH -p debug
+  #SBATCH --nodes=1
+  #SBATCH --tasks-per-node=1
+  #SBATCH --cpus-per-task=32
+  #SBATCH -t 00:10:00
+  #SBATCH -C haswell
+  #SBATCH -J pytest
+  #SBATCH -e pytest.err
+  #SBATCH -o pytest.out
+  #SBATCH --image=continuumio/anaconda3:2020.11
+  module load python/3.8-anaconda-2020.11
+  python -m pytest --runcori
+
+The check the output in ``pytest.out`` to see that all the tests passed.
+
 Writing Tests
 ~~~~~~~~~~~~~
 
@@ -145,6 +176,8 @@ Tests should be added to ``tests`` directory. If writing component to
 use for testing that should go into ``tests/components`` and any
 executable should go into ``tests/bin``.
 
+.. _continuous integration:
+
 Continuous Integration (CI)
 ---------------------------
 
@@ -220,3 +253,6 @@ https://github.com/HPC-SimTools/IPS-framework/releases
 We will publish a release candidate versions for any major or minor
 release before the full release to allow feedback from users. Patch
 versions will not normally have an release candidate.
+
+Before a release is finalized the :ref:`Cori only tests<cori-tests>`
+should be run.

doc/user_guides/dask.rst

@@ -54,6 +54,7 @@ simulation log is shown:
 .. literalinclude::  ../examples/dask/simulation_log.json
    :language: JSON
 
+.. _dask_shifter:
 
 Running dask in shifter
 -----------------------

doc/user_guides/nersc_conda.rst

@@ -172,8 +172,8 @@ Miniconda or Anaconda can can be installed to your home directory for easy devel
 .. code-block:: bash
 
    wget https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh
-   bash Anaconda3-2019.10-Linux-x86_64.sh -b -p $HOME/miniconda3
-   source $HOME/miniconda3/bin/activate
+   bash Anaconda3-2019.10-Linux-x86_64.sh -b -p $HOME/anaconda3
+   source $HOME/anaconda3/bin/activate
    conda install netcdf4
    MPICC="$(which cc) --shared" python -m pip install --no-binary mpi4py mpi4py
    python -m pip install dask-mpi

setup.py

@@ -54,7 +54,8 @@
         'Programming Language :: Python :: 3.8',
         'Programming Language :: Python :: 3.9',
         "License :: OSI Approved :: BSD License",
-        "Operating System :: OS Independent",
+        "Operating System :: MacOS :: MacOS X",
+        "Operating System :: POSIX :: Linux",
     ],
     python_requires='>=3.6',
     zip_safe=True