Release 0.9.3

tennlee · web-flow · commit 3316dba7e75f · 2024-07-09T15:24:10.000+10:00
Release 0.9.3
diff --git a/.binder/requirements.txt b/.binder/requirements.txt
@@ -33,7 +33,7 @@ bleach==6.1.0
     # via nbconvert
 bottleneck==1.3.8
     # via scores (pyproject.toml)
-certifi==2024.2.2
+certifi==2024.7.4
     # via
     #   httpcore
     #   httpx
diff --git a/.github/workflows/python-app.yml b/.github/workflows/python-app.yml
@@ -41,4 +41,12 @@ jobs:
       with:
         flag-name: run-${{ join(matrix.*, '-') }}
         parallel: true
+  finish:
+    needs: build
+    runs-on: ubuntu-latest
+    steps:
+    - name: Indicate Coveralls Parallel Finished
+      uses: coverallsapp/github-action@v2.3.0
+      with: 
+        parallel-finished: true
   
diff --git a/CITATION.cff b/CITATION.cff
@@ -32,13 +32,13 @@ authors:
 - given-names: John 
   family-names: Sharples  
 
-title: "scores: A Python package for verifying and evaluating models and predictions with xarray and pandas"  
+title: "scores: A Python package for verifying and evaluating models and predictions with xarray"  
 
 preferred-citation:
   type: article
   doi: "10.48550/arXiv.2406.07817"
   journal: "arXiv"
-  title: "scores: A Python package for verifying and evaluating models and predictions with xarray and pandas"
+  title: "scores: A Python package for verifying and evaluating models and predictions with xarray"
   authors:
   - given-names: Tennessee 
     family-names: Leeuwenburg
@@ -70,4 +70,4 @@ preferred-citation:
     family-names: Griffiths
   - given-names: John 
     family-names: Sharples  
-  year: 2024
+  year: 2024
diff --git a/README.md b/README.md
@@ -75,5 +75,5 @@ If you use `scores` for a published work, we would appreciate you citing our arX
 
 Leeuwenburg, T., Loveday, N., Ebert, E. E., Cook, H., Khanarmuei, M., Taggart, R. J., Ramanathan, N., 
 Carroll, M., Chong, S., Griffiths, A., & Sharples, J. (2024). *scores: A Python package for verifying and 
-evaluating models and predictions with xarray and pandas*. arXiv. [https://doi.org/10.48550/arXiv.2406.07817](https://doi.org/10.48550/arXiv.2406.07817)
+evaluating models and predictions with xarray*. arXiv. [https://doi.org/10.48550/arXiv.2406.07817](https://doi.org/10.48550/arXiv.2406.07817)
 
diff --git a/docs/api.md b/docs/api.md
@@ -20,7 +20,7 @@
 .. autofunction:: scores.continuous.murphy_thetas
 .. autofunction:: scores.continuous.flip_flop_index
 .. autofunction:: scores.continuous.flip_flop_index_proportion_exceeding
-.. autofunction:: scores.continuous.correlation
+.. autofunction:: scores.continuous.correlation.pearsonr
 .. autofunction:: scores.continuous.multiplicative_bias
 .. autofunction:: scores.continuous.isotonic_fit
 .. autofunction:: scores.continuous.consistent_expectile_score
diff --git a/docs/conf.py b/docs/conf.py
@@ -9,7 +9,7 @@
 
 project = "scores"
 copyright = "Licensed under Apache 2.0 - https://www.apache.org/licenses/LICENSE-2.0"
-release = "0.9.2"
+release = "0.9.3"
 
 version = __version__
 
diff --git a/docs/included.md b/docs/included.md
@@ -94,7 +94,7 @@
   - 
     [Ehm et al. (2016) - Corollary 2 (p.521)](https://doi.org/10.1111/rssb.12154); [Taggart (2022) - Corollary 5.6](https://doi.org/10.1214/21-ejs1957)
 * - Pearson's Correlation Coefficient
-  - [API](api.md#scores.continuous.correlation)
+  - [API](api.md#scores.continuous.correlation.pearsonr)
   - [Tutorial](project:./tutorials/Pearsons_Correlation.md)
   - [Wikipedia](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)
 * - Pinball Loss, *see Quantile Loss*
diff --git a/docs/index.md b/docs/index.md
@@ -21,4 +21,5 @@ data
 tutorials/Tutorial_Gallery
 related_works
 maintainer
+release_notes
 ```
diff --git a/docs/installation.md b/docs/installation.md
@@ -1,6 +1,12 @@
 # Detailed Installation Guide
 
-This page describes the most common installation options for `scores`. Expert users of pip and conda will note that there are more variations possible.
+## Overview
+
+This page describes:
+
+- Setting up a virtual environment.
+- The most common installation options for `scores`. (Expert users of pip and conda will note that more variations are possible).
+- An advanced installation option for Jupyter Notebook, for users who wish to separate the Jupyter environment and the `scores` execution environment.
 
 ## Setting up a Virtual Environment
 
@@ -70,7 +76,7 @@ Use this for running tutorials using `scores`, but when you don't need or want d
 
 Installs:
 * Mathematical API functions and libraries
-* Jupyter Lab, Plotly, and libraries for reading data, so that the tutorial notebooks can be run
+* JupyterLab, Plotly, and libraries for reading data, so that the tutorial notebooks can be run
 
 #### From a Local Git Repository
 
@@ -92,4 +98,20 @@ Installs:
 pip install -e .[maintainer]
 ```
 
+## Jupyter Notebook - Advanced Installation Option
+
+The `scores` "all" and "tutorial" installation options include the JupyterLab software, which can be used to run the tutorials and/or execute `scores` code within a Jupyter environment. 
+
+Some users may wish to separate the Jupyter environment and the `scores` execution environment. One way to achieve this is by creating a new `scores` virtual environment (using one of the [above options](#setting-up-a-virtual-environment)) and registering it as a new kernel within the Jupyter environment. You can then run the tutorials and/or execute `scores` code within the kernel. Registering the kernel can be done as follows:
+
+1. Determine the "prefix" of the Jupyter environment. 
+2. Choose a name to use for a new kernel.
+3. Activate the `scores` virtual environment which will be used as the kernel.
+4. Execute the registration command.
+
+A sample command to register a new kernel is:
+
+`python -m ipykernel install --user --prefix=<path-to-server-environment> --name=<pick-any-name-here>`
+
+[https://jupyter-tutorial.readthedocs.io/en/24.1.0/kernels/install.html](https://jupyter-tutorial.readthedocs.io/en/24.1.0/kernels/install.html) provides additional technical details regarding the registration of kernels.
 
diff --git a/docs/key_features.md b/docs/key_features.md
@@ -25,4 +25,4 @@
 - Easy to integrate and use in a wide variety of environments. It has been used on workstations, servers and in high performance computing (supercomputing) environments. 
 - Maintains 100% automated test coverage.
 - Uses [Dask](http://dask.pydata.org) for scaling and performance.
-- Expanding support for [pandas](https://pandas.pydata.org/).
+- [Some metrics](included.md#pandas) work with [pandas](https://pandas.pydata.org/) and we aim to expand this capability. 
diff --git a/docs/maintainer.md b/docs/maintainer.md
@@ -39,6 +39,24 @@ Information relevant for package maintenance
 5. Run hatch publish
 6. Uninstall the test scores, re-install the now-updated package, do a little testing
 
+## This section covers how to format release notes
+
+```
+# Release Notes (What's New)
+
+## Version X.Y.Z (Month Day, Year) e.g. "Version 0.9.3 (July 7, 2024)"
+
+For the full details of all changes in this release, see the [GitHub commit history](https://github.com/nci/scores/compare/(X-1).(Y-1).(Z-1)...X.Y.Z). Below are the changes we think users may wish to be aware of."
+
+### Features
+### Breaking Changes
+### Deprecations
+### Bug Fixes
+### Documentation
+### Internal Changes
+
+For each entry: "Brief description. See [PR #ABC](add link) by [@username](add link)."
+```
 
 ## This section covers how to build the documentation locally 
 (Readthedocs should update automatically from a GitHub Action)
diff --git a/docs/release_notes.md b/docs/release_notes.md
@@ -0,0 +1,19 @@
+# Release Notes (What's New)
+
+## Version 0.9.3 (July 9, 2024)
+
+For the full details of all changes in this release, see the [GitHub commit history](https://github.com/nci/scores/compare/0.9.2...0.9.3). Below are the changes we think users may wish to be aware of.
+
+### Breaking Changes
+
+- Renamed and relocated function `scores.continuous.correlation` to `scores.continuous.correlation.pearsonr`. See [PR #583](https://github.com/nci/scores/pull/583) by [@nicholasloveday](https://github.com/nicholasloveday). 
+
+### Documentation
+
+- Added "Dimension Handling" tutorial, which describes reducing and preserving dimensions. See [PR #589](https://github.com/nci/scores/pull/589) by [@nicholasloveday](https://github.com/nicholasloveday).
+- Updated "Detailed Installation Guide" with information on installing kernels in a Jupyter environment. See [PR #586](https://github.com/nci/scores/pull/586) by [@tennlee](https://github.com/tennlee) and [PR #587](https://github.com/nci/scores/pull/587) by [@Steph-Chong](https://github.com/Steph-Chong).
+
+### Internal Changes
+
+- Introduced pinned versions for dependencies on main. See [PR #580](https://github.com/nci/scores/pull/580)  by [@tennlee](https://github.com/tennlee).
+
diff --git a/src/scores/__init__.py b/src/scores/__init__.py
@@ -13,7 +13,7 @@
 import scores.sample_data
 import scores.stats.statistical_tests  # noqa: F401
 
-__version__ = "0.9.2"
+__version__ = "0.9.3"
 
 __all__ = [
     "scores.categorical",
diff --git a/src/scores/continuous/__init__.py b/src/scores/continuous/__init__.py
@@ -2,6 +2,7 @@
 Import the functions from the implementations into the public API
 """
 
+import scores.continuous.correlation  # noqa: F401
 from scores.continuous.consistent_impl import (
     consistent_expectile_score,
     consistent_huber_score,
@@ -15,7 +16,6 @@
 from scores.continuous.quantile_loss_impl import quantile_score
 from scores.continuous.standard_impl import (
     additive_bias,
-    correlation,
     mae,
     mean_error,
     mse,
diff --git a/src/scores/continuous/correlation/__init__.py b/src/scores/continuous/correlation/__init__.py
@@ -0,0 +1,7 @@
+"""
+Import the functions from the implementations into the public API
+"""
+
+from scores.continuous.correlation.correlation_impl import pearsonr
+
+__all__ = ["pearsonr"]
diff --git a/src/scores/continuous/correlation/correlation_impl.py b/src/scores/continuous/correlation/correlation_impl.py
@@ -0,0 +1,60 @@
+"""
+This module contains functions to calculate the correlation.
+"""
+
+from typing import Optional
+
+import xarray as xr
+
+import scores.utils
+from scores.typing import FlexibleDimensionTypes
+
+
+def pearsonr(
+    fcst: xr.DataArray,
+    obs: xr.DataArray,
+    *,  # Force keywords arguments to be keyword-only
+    reduce_dims: Optional[FlexibleDimensionTypes] = None,
+    preserve_dims: Optional[FlexibleDimensionTypes] = None,
+) -> xr.DataArray:
+    """
+    Calculates the Pearson's correlation coefficient between two xarray DataArrays
+
+    .. math::
+        \\rho = \\frac{\\sum_{i=1}^{n}{(x_i - \\bar{x})(y_i - \\bar{y})}}
+        {\\sqrt{\\sum_{i=1}^{n}{(x_i-\\bar{x})^2}\\sum_{i=1}^{n}{(y_i - \\bar{y})^2}}}
+
+    where:
+        - :math:`\\rho` = Pearson's correlation coefficient
+        - :math:`x_i` = the values of x in a sample (i.e. forecast values)
+        - :math:`\\bar{x}` = the mean value of the forecast sample
+        - :math:`y_i` = the values of y in a sample (i.e. observed values)
+        - :math:`\\bar{y}` = the mean value of the observed sample value
+
+    Args:
+        fcst: Forecast or predicted variables
+        obs: Observed variables.
+        reduce_dims: Optionally specify which dimensions to reduce when
+            calculating the Pearson's correlation coefficient.
+            All other dimensions will be preserved.
+        preserve_dims: Optionally specify which dimensions to preserve when
+            calculating the Pearson's correlation coefficient. All other dimensions will
+            be reduced. As a special case, 'all' will allow all dimensions to be
+            preserved. In this case, the result will be in the same shape/dimensionality
+            as the forecast, and the errors will be the absolute error at each
+            point (i.e. single-value comparison against observed), and the
+            forecast and observed dimensions must match precisely.
+    Returns:
+        xr.DataArray: An xarray object with Pearson's correlation coefficient values
+
+    Note:
+        This function isn't set up to take weights.
+
+    Reference:
+        https://en.wikipedia.org/wiki/Pearson_correlation_coefficient
+    """
+    reduce_dims = scores.utils.gather_dimensions(
+        fcst.dims, obs.dims, reduce_dims=reduce_dims, preserve_dims=preserve_dims
+    )
+
+    return xr.corr(fcst, obs, reduce_dims)
diff --git a/src/scores/continuous/standard_impl.py b/src/scores/continuous/standard_impl.py
@@ -199,50 +199,6 @@ def mae(
     return _ae  # type: ignore
 
 
-def correlation(
-    fcst: xr.DataArray,
-    obs: xr.DataArray,
-    *,  # Force keywords arguments to be keyword-only
-    reduce_dims: Optional[FlexibleDimensionTypes] = None,
-    preserve_dims: Optional[FlexibleDimensionTypes] = None,
-) -> xr.DataArray:
-    """
-    Calculates the Pearson's correlation coefficient between two xarray DataArrays
-
-    .. math::
-        \\rho = \\frac{\\sum_{i=1}^{n}{(x_i - \\bar{x})(y_i - \\bar{y})}}
-        {\\sqrt{\\sum_{i=1}^{n}{(x_i-\\bar{x})^2}\\sum_{i=1}^{n}{(y_i - \\bar{y})^2}}}
-
-    where:
-        - :math:`\\rho` = Pearson's correlation coefficient
-        - :math:`x_i` = the values of x in a sample (i.e. forecast values)
-        - :math:`\\bar{x}` = the mean value of the forecast sample
-        - :math:`y_i` = the values of y in a sample (i.e. observed values)
-        - :math:`\\bar{y}` = the mean value of the observed sample value
-
-    Args:
-        fcst: Forecast or predicted variables
-        obs: Observed variables.
-        reduce_dims: Optionally specify which dimensions to reduce when
-            calculating the Pearson's correlation coefficient.
-            All other dimensions will be preserved.
-        preserve_dims: Optionally specify which dimensions to preserve when
-            calculating the Pearson's correlation coefficient. All other dimensions will
-            be reduced. As a special case, 'all' will allow all dimensions to be
-            preserved. In this case, the result will be in the same shape/dimensionality
-            as the forecast, and the errors will be the absolute error at each
-            point (i.e. single-value comparison against observed), and the
-            forecast and observed dimensions must match precisely.
-    Returns:
-        xr.DataArray: An xarray object with Pearson's correlation coefficient values
-    """
-    reduce_dims = scores.utils.gather_dimensions(
-        fcst.dims, obs.dims, reduce_dims=reduce_dims, preserve_dims=preserve_dims
-    )
-
-    return xr.corr(fcst, obs, reduce_dims)
-
-
 def mean_error(
     fcst: XarrayLike,
     obs: XarrayLike,
diff --git a/src/scores/probability/roc_impl.py b/src/scores/probability/roc_impl.py
@@ -25,11 +25,11 @@ def roc_curve_data(  # pylint: disable=too-many-arguments
 ) -> xr.Dataset:
     """
     Calculates data required for plotting a Receiver (Relative) Operating Characteristic (ROC)
-      curve including the AUC. The ROC curve is used as a way to measure the discrimination
-      ability of a particular forecast.
+    curve, including the area under the curve (AUC). The ROC curve is used as a way to measure
+    the discrimination ability of a particular forecast.
 
-      The AUC is the probability that the forecast probability of a random event is higher
-        than the forecast probability of a random non-event.
+    The AUC is the probability that the forecast probability of a random event is higher
+    than the forecast probability of a random non-event.
 
     Args:
         fcst: An array of probabilistic forecasts for a binary event in the range [0, 1].
@@ -66,6 +66,14 @@ def roc_curve_data(  # pylint: disable=too-many-arguments
         `POD` and `POFD` have dimensions `dims` + 'threshold', while `AUC` has
         dimensions `dims`.
 
+    Raises:
+        ValueError: if `fcst` contains values outside of the range [0, 1]
+        ValueError: if `obs` contains non-nan values not in the set {0, 1}
+        ValueError: if 'threshold' is a dimension in `fcst`.
+        ValueError: if values in `thresholds` are not monotonic increasing or are outside
+          the range [0, 1]
+
+
     Notes:
         The probabilistic `fcst` is converted to a deterministic forecast
         for each threshold in `thresholds`. If a value in `fcst` is greater
@@ -78,12 +86,6 @@ def roc_curve_data(  # pylint: disable=too-many-arguments
         Ideally concave ROC curves should be generated rather than traditional
         ROC curves.
 
-    Raises:
-        ValueError: if `fcst` contains values outside of the range [0, 1]
-        ValueError: if `obs` contains non-nan values not in the set {0, 1}
-        ValueError: if 'threshold' is a dimension in `fcst`.
-        ValueError: if values in `thresholds` are not monotonic increasing or are outside
-          the range [0, 1]
     """
     if check_args:
         if fcst.max().item() > 1 or fcst.min().item() < 0:
diff --git a/tests/continuous/test_correlation.py b/tests/continuous/test_correlation.py
diff --git a/tests/continuous/test_standard.py b/tests/continuous/test_standard.py
diff --git a/tutorials/Dimension_Handling.ipynb b/tutorials/Dimension_Handling.ipynb
diff --git a/tutorials/Mean_Absolute_Error.ipynb b/tutorials/Mean_Absolute_Error.ipynb
diff --git a/tutorials/Mean_Squared_Error.ipynb b/tutorials/Mean_Squared_Error.ipynb
diff --git a/tutorials/Pearsons_Correlation.ipynb b/tutorials/Pearsons_Correlation.ipynb
diff --git a/tutorials/Root_Mean_Squared_Error.ipynb b/tutorials/Root_Mean_Squared_Error.ipynb
diff --git a/tutorials/Tutorial_Gallery.ipynb b/tutorials/Tutorial_Gallery.ipynb
diff --git a/tutorials/Weighting_Results.ipynb b/tutorials/Weighting_Results.ipynb

Original file line number	Diff line number	Diff line change
`@@ -94,7 +94,7 @@`
`94`	`94`	`-`
`95`	`95`	`[Ehm et al. (2016) - Corollary 2 (p.521)](https://doi.org/10.1111/rssb.12154); [Taggart (2022) - Corollary 5.6](https://doi.org/10.1214/21-ejs1957)`
`96`	`96`	`* - Pearson's Correlation Coefficient`
`97`		`- - [API](api.md#scores.continuous.correlation)`
	`97`	`+ - [API](api.md#scores.continuous.correlation.pearsonr)`
`98`	`98`	`- [Tutorial](project:./tutorials/Pearsons_Correlation.md)`
`99`	`99`	`- [Wikipedia](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)`
`100`	`100`	`* - Pinball Loss, see Quantile Loss`