Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update CI image and add mkmf for fre make testing #245

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 22 additions & 34 deletions .github/workflows/create_test_conda_env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,66 +5,54 @@ on: [push]
jobs:
build-linux:
runs-on: ubuntu-latest
container:
image: ghcr.io/noaa-gfdl/fre-cli:miniconda24.7.1_gcc14.2.0
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '>=3.9'

- name: Add conda to system path
run: |
# $CONDA is an env var pointing to root of miniconda dir
echo $CONDA/bin >> $GITHUB_PATH

submodules: recursive
- name: Create fre-cli environment
run: |
# create environment containing all dependencies
# the env cannot be explicitly activated in github CI/CD
conda env create -f environment.yml --name fre-cli

# add conda env's executables to github's PATH equiv.
# sets CONDA to wherever it may be on the image
source /root/.bashrc

# add conda env's executables and mkmf to github's PATH equiv.
echo $CONDA/envs/fre-cli/bin >> $GITHUB_PATH

echo $PWD/mkmf/bin >> $GITHUB_PATH

# use *conda environment's pip* to install fre-cli
# called w/ full path to conda's python for explicitness
# called as a module (-m pip) for explicitness
$CONDA/envs/fre-cli/bin/python -m pip install --prefix $CONDA/envs/fre-cli .
$CONDA/envs/fre-cli/bin/python -m pip install --prefix $CONDA/envs/fre-cli .

- name: Run pytest in fre-cli environment
run: |
# try to make sure the right things are in GITHUB_PATH
echo $CONDA/envs/fre-cli/bin >> $GITHUB_PATH

# are we talking to the right python?
which python
python --version
$CONDA/envs/fre-cli/bin/python --version

# add spack installed binaries to front of path so that
# conda's netcdf/hdf5 installs don't break compilation tests
export path_save=$PATH
export PATH="/opt/views/view/bin:$PATH"

# run pytest
pytest --junit-xml=pytest_results.xml --config-file=fre/pytest.ini --cov-config=fre/coveragerc --cov-report=xml --cov=fre fre/

# install genbadge to generate coverage badge based on xml

# restore original path and install genbadge to generate coverage badge based on xml
export PATH="$path_save"
pip install genbadge
genbadge coverage -v -i coverage.xml -o docs/cov_badge.svg
genbadge tests -v -i pytest_results.xml -o docs/pytest_badge.svg

- name: Run pylint in fre-cli environment
run: |
# try to make sure the right things are in GITHUB_PATH
echo $CONDA/envs/fre-cli/bin >> $GITHUB_PATH

# are we talking to the right python?
which python
python --version
$CONDA/envs/fre-cli/bin/python --version

# run pylint, ignored modules avoid warnings arising from code internal to those modules
# run pylint, ignored modules avoid warnings arising from code internal to those modules
pylint --max-args 6 -ry --ignored-modules netCDF4,cmor fre/ || echo "pylint returned non-zero exit code. preventing workflow from dying with this echo."

- name: Install Sphinx and Build Documentation
run: |
pip install sphinx renku-sphinx-theme sphinx-rtd-theme
pip install sphinx renku-sphinx-theme sphinx-rtd-theme
pip install --upgrade sphinx-rtd-theme
sphinx-apidoc --output-dir docs fre/ --separate
sphinx-build docs build
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "mkmf"]
path = mkmf
url = https://github.com/NOAA-GFDL/mkmf
6 changes: 6 additions & 0 deletions fre/make/tests/null_example/platforms.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,9 @@ platforms:
container: True
containerBuild: "podman"
containerRun: "apptainer"
- name: ci.gnu
compiler: gnu
fc: mpifort
cc: mpicc
mkTemplate: /__w/fre-cli/fre-cli/mkmf/templates/linux-ubuntu-xenial-gnu.mk
modelRoot: ${HOME}/fremake_canopy/test
13 changes: 13 additions & 0 deletions fre/make/tests/test_fre_make_run_fremake.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
''' test "fre make" calls '''

from click.testing import CliRunner

from fre import fre

runner = CliRunner()

## assumes you're running pytest from repo root dir
def test_fre_make_run_fremake_null_model():
''' run fre make with run-fremake subcommand and build the null model experiment with gnu'''
result = runner.invoke(fre.fre, args=["make", "run-fremake", "-y", "fre/make/tests/null_example/null_model.yaml", "-p", "ci.gnu", "-t", "debug"])
assert result.exit_code == 0
1 change: 1 addition & 0 deletions mkmf
Submodule mkmf added at 9830f1
Loading