Transition pynio to cfgrib for wgrib2 engine. #263

christinaholtNOAA · 2025-12-02T16:58:01Z

There are no significant changes here, only those related to typing, formatting, and linting.

-Original file line number
+Diff line change
@@ -0,0 +1,3 @@
+    tests/data/wrfnat_hrconus_16.grib2 filter=lfs diff=lfs merge=lfs -text
+    tests/data/wrfprs_hrconus_16.grib2 filter=lfs diff=lfs merge=lfs -text
+    tests/data/wrf* filter=lfs diff=lfs merge=lfs -text

-Original file line number
+Diff line change
@@ Expand Up / @@ -12,21 +12,20 @@ jobs: @@
         runs-on: ubuntu-latest
         defaults:
           run:
-            shell: bash -l {0}
+            shell: bash -el {0}
         steps:
           - name: Checkout repo
-            uses: actions/checkout@v2
+            uses: actions/checkout@v6
             with:
               lfs: true
-          - name: Install Micromamba with pygraf environment
-            uses: mamba-org/setup-micromamba@v1
+          - name: Install Miniforge with pygraf environment
+            uses: conda-incubator/setup-miniconda@v3
             with:
               environment-file: environment.yml
-              cache-downloads: true
-              cache-environment: true
-          - name: Lint code
-            run: find . -type f -name "*.py" | xargs pylint
-            shell: bash -el {0}
+              miniforge-version: 25.11.0-0
+          - name: Install dev pkgs
+            run: make devenv
           - name: Test code
-            run: python -m pytest --nat-file tests/data/wrfnat_hrconus_07.grib2 --prs-file tests/data/wrfprs_hrconus_07.grib2 --ignore=tests/test_hrrr_maps.py
-            shell: bash -el {0}
+            run: |
+              conda activate pygraf
+              make test

-Original file line number
+Diff line change
@@ -0,0 +1,30 @@
+    TARGETS = devenv env format lint test typecheck unittest
+    DEVPKGS = $(shell cat devpkgs)
+    ENVNAME = pygraf
+    .PHONY: $(TARGETS)
+    devenv: env
+    	mamba install -y -n $(ENVNAME) $(DEVPKGS)
+    env:
+    	mamba env create -y -f environment.yml
+    format:
+    	@./format
+    lint:
+    	ruff check .
+    test: lint typecheck unittest
+    typecheck:
+    	mypy --install-types --non-interactive .
+    unittest:
+    	pytest --cov -k "not hrrr_maps" -n 4 .
+    memtest:
+    	pytest --memray -k "not hrrr" .

-Original file line number
+Diff line change
@@ -1,7 +1,5 @@
     # ADB Graphics Creation for UPP Model Output
-    > Note: This repository is under heavy development. Use at your own risk!
     This repository houses a Python-based implementation of the graphics package
     that is responsible for generating maps for the RAP/HRRR/FV3/RRFS data. It has
     replaced NCL as the real-time graphics creation package at NOAA GSL for maps and
@@ Expand Down Expand Up / @@ -40,10 +38,30 @@ activate this environment, do the following: @@
     ```
     module use -a /contrib/miniconda3/modulefiles
-    module load miniconda3
+    module load miniconda3/25.11.0
     conda activate pygraf
     ```
+    This environment contains the necessary development packages.
+    ## Installing with conda
+    Pygraf comes with an `environment.yml` file for use with any conda installation. Ensure the conda base
+    environment is activated, and run the following command to create a `pygraf` environment suitable
+    for creating graphics:
+    ```
+    cd pygraf
+    make env
+    ```
+    For developers who want to run the test suite before contributing new changes to the repository,
+    additional development packages are required. To install those, run:
+    ```
+    cd pygraf
+    make devenv
+    ```
     ## Stage data
@@ Expand Down Expand Up / @@ -143,7 +161,7 @@ python create_graphics.py \ @@
              --tiles full,ATL,CA-NV,CentralCA
     ```
-    NOTE: The graphics already run as a workflow step in the RRFS Retros! They may be
+    > NOTE: The graphics already run as a workflow step in the RRFS Retros! They may be
     zipped by default, so you can unzip those files to see your images on disk.
     ### Creating Skew-T Diagrams
@@ Expand Down Expand Up / @@ -274,16 +292,58 @@ guidelines: @@
     - All code must pass tests, and tests must be updated to accommodate new code.
     - Style beyond linting:
       - Alphabetize lists (anywhere another order is not more obvious to everyone)
-      - A single white space line before and after comments.
-      - A single white space after each method/function. Two after classes.
-      - Lists are maintained with each item on a single line followed by a comma,
-      even the last item.
     This repository is using a minor variation on GitLab flow, requiring new work be
     contributed via Pull Request from a branch with reviewers (required). Releases
     will be handled with tags (as opposed to branches, in the original GitLab flow),
     and will be marked as versions with v[major].[minor].[update].
+    # Running Tests
+    GitHub Actions is configured to run several code quality checks, including linting, formatting, and
+    sorting with `ruff`, type checking with `mypy`, and unit tests with `pytest`. To perform the same
+    checks locally, developers can run:
+    ```
+    (pygraf) $ make format && make test
+    ```
+    # Working with ecCodes for grib2
+    Two command line utilities are available in the conda environment that will help navigate the
+    ecCodes interpretation of a grib2 file. `grib_ls` gives a single-line record listing of the entire file by
+    default, while `grib_dump` provides all the metadata for each record.
+    Documentation is available from ECMWF:
+    *  https://confluence.ecmwf.int/display/ECC/grib_ls
+    *  https://confluence.ecmwf.int/display/ECC/grib_dump
+    There are many examples in the documentation, but here are a couple that could help with `pygraf`
+    specifically.
+    Show the common `pygraf` parameters for the 6th record:
+    ```
+    $ grib_ls -p shortName,parameterNumber,typeOfLevel,stepType,level -w count=6 hrrr.t05z.wrfnatf08.grib2
+    hrrr.t05z.wrfnatf08.grib2
+    shortName        parameterNumber  typeOfLevel      stepType         level
+    grle             32               hybrid           instant          1
+    ```
+    The `count` parameter is nice to use in conjunction with `wgrib2` output where that tool may show
+    the information needed to identify a variable where the `shortName` from ecCodes may be "unknown".
+    To see all available information from that record:
+    ```
+    grib_dump -w count=6 hrrr.t05z.wrfnatf08.grib2
+    ```
+    > NOTE: When using the `-w` flag with items other than `count`, multiple records may be included in
+    the output.
     # Contact
     | Name | Email |
@@ Expand Down @@

-Original file line number
+Diff line change
@@ -1,109 +1,109 @@
-    # pylint: disable=unused-argument,invalid-name
-    '''
+    """
     This module contains functions for converting the units of a field. The
     interface requires a single atmospheric field in a Numpy array, and returns the
     converted values as output.
-    '''
+    """
-    import numpy as np
+    from numpy import ndarray
+    from xarray import DataArray
+    from xarray.ufuncs import sqrt, square
-    def k_to_c(field, **kwargs):
-        ''' Conversion from Kelvin to Celsius '''
+    def k_to_c(field: ndarray, **_kwargs):
+        """Conversion from Kelvin to Celsius."""
         return field - 273.15
-    def k_to_f(field, **kwargs):
-        ''' Conversion from Kelvin to Farenheit '''
+    def k_to_f(field: ndarray, **_kwargs):
+        """Conversion from Kelvin to Fahrenheit."""
+        return (field - 273.15) * 9 / 5 + 32
-        return (field - 273.15) * 9/5 + 32
-    def kgm2_to_in(field, **kwargs):
-        ''' Conversion from kg per m^2 to inches '''
+    def kgm2_to_in(field: ndarray, **_kwargs):
+        """Conversion from kg per m^2 to inches."""
         return field * 0.03937
-    def magnitude(a, b, **kwargs):
-        ''' Return the magnitude of vector components '''
+    def magnitude(a: DataArray, b: DataArray, **_kwargs) -> DataArray:
+        """Return the magnitude of vector components."""
-        return np.sqrt(np.square(a) + np.square(b))
+        return DataArray(sqrt(square(a) + square(b)))
-    def m_to_dm(field, **kwargs):
-        ''' Conversion from meters to decameters '''
+    def m_to_dm(field: ndarray, **_kwargs):
+        """Conversion from meters to decameters."""
-        return field / 10.
+        return field / 10.0
-    def m_to_in(field, **kwargs):
-        ''' Conversion from meters to inches '''
+    def m_to_in(field: ndarray, **_kwargs):
+        """Conversion from meters to inches."""
         return field * 39.3701
-    def m_to_kft(field, **kwargs):
-        ''' Conversion from meters to kilofeet '''
+    def m_to_kft(field: ndarray, **_kwargs):
+        """Conversion from meters to kilofeet."""
         return field / 304.8
-    def m_to_mi(field, **kwargs):
-        ''' Conversion from meters to miles '''
+    def m_to_mi(field: ndarray, **_kwargs):
+        """Conversion from meters to miles."""
         return field / 1609.344
-    def ms_to_kt(field, **kwargs):
-        ''' Conversion from m s-1 to knots '''
+    def ms_to_kt(field: ndarray, **_kwargs):
+        """Conversion from m s-1 to knots."""
         return field * 1.9438
-    def pa_to_hpa(field, **kwargs):
-        ''' Conversion from Pascals to hectopascals '''
+    def pa_to_hpa(field: ndarray, **_kwargs):
+        """Conversion from Pascals to hectopascals."""
-        return field / 100.
+        return field / 100.0
-    def percent(field, **kwargs):
-        ''' Conversion from values between 0 - 1 to percent '''
+    def percent(field: ndarray, **_kwargs):
+        """Conversion from values between 0 - 1 to percent."""
-        return field * 100.
+        return field * 100.0
-    def to_micro(field, **kwargs):
-        ''' Convert field to micro '''
+    def sden_to_slr(field: ndarray, **_kwargs):
+        """Convert snow density (kg m-3) to snow-liquid ratio."""
-        return field * 1E6
+        return 1000.0 / field
-    def to_micrograms_per_m3(field, **kwargs):
-        ''' Convert field to micrograms per cubic meter '''
+    def to_micro(field: ndarray, **_kwargs):
+        """Convert field to micro."""
-        return field * 1E9
+        return field * 1e6
-    def vvel_scale(field, **kwargs):
-        ''' Scale vertical velocity for plotting  '''
+    def to_micrograms_per_m3(field: ndarray, **_kwargs):
+        """Convert field to micrograms per cubic meter."""
-        return field * -10
+        return field * 1e9
-    def vort_scale(field, **kwargs):
-        ''' Scale vorticity for plotting  '''
+    def vvel_scale(field: ndarray, **_kwargs):
+        """Scale vertical velocity for plotting."""
-        return field / 1E-05
+        return field * -10
-    def weasd_to_1hsnw(field, **kwargs):
-        ''' Conversion from snow wter equiv to snow (10:1 ratio) '''
+    def vort_scale(field: ndarray, **_kwargs):
+        """Scale vorticity for plotting."""
-        return field * 10.
+        return field / 1e-05
-    def sden_to_slr(field, **kwargs):
-        ''' Convert snow density (kg m-3) to snow-liquid ratio '''
+    def weasd_to_1hsnw(field: ndarray, **_kwargs):
+        """Conversion from snow water equiv to snow (10:1 ratio)."""
-        return 1000. / field
+        return field * 10.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transition pynio to cfgrib for wgrib2 engine. #263

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

christinaholtNOAA Dec 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Transition pynio to cfgrib for wgrib2 engine. #263

Are you sure you want to change the base?

Uh oh!

Transition pynio to cfgrib for wgrib2 engine. #263

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

christinaholtNOAA Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!