Release 2.0 (#170)

* Update changelog

* Fix dependencies

* Update notices on readme

* Add py3.12 to changelog

* Make newest cads-api-client version work

* Bump version to 2.0.0

* Drop py38 add py312 for the testing workflow

* Remove reference to old cdsapi package

* Turns out the the cads-api-client still needs the cdsapi

* Patch new LegacyApiClient class instead of old client

* Fix API again, add 3.13 support.

Author: BSchilperoort

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 2.0.0b2
+current_version = 2.0.0
 
 [comment]
 comment = The contents of this file cannot be merged with that of pyproject.toml until https://github.com/c4urself/bump2version/issues/42 is resolved

.github/workflows/test_and_build.yml

@@ -12,7 +12,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
     name: Run tests for ${{ matrix.python-version }}
     steps:
     - uses: actions/checkout@v3

CITATION.cff

@@ -97,4 +97,4 @@ license: Apache-2.0
 message: "If you use this software, please cite it using these metadata."
 repository-code: "https://github.com/ewatercycle/era5cli"
 title: era5cli
-version: "2.0.0b2"
+version: "2.0.0"

README.md

@@ -1,22 +1,3 @@
-<img align="right" width="150" alt="Logo" src="docs/assets/era5cli_logo_colors_border.png">
-
-> [!IMPORTANT]
-> The old Climate Data Store (CDS) will be shut down on 3 September 2024.
-> All era5cli versions up to v1.4.2 will no longer work.
->
-> For more information see: https://forum.ecmwf.int/t/the-new-climate-data-store-beta-cds-beta-is-now-live/3315
->
-> 
-> To continue using era5cli, you will need to re-register at ECMWF and get a new API key,
-> and transition to the era5cli v2 beta. This can be installed with:
-> `pip install era5cli==2.0.0b2` 
-
-> [!WARNING]
-> netCDF files from the new Climate Data Store Beta are not formatted the same as the
-> old CDS. Some variables might be missing.
-> See the open issue [here](https://github.com/eWaterCycle/era5cli/issues/165), as well as the [ECMWF discussion forum](https://forum.ecmwf.int/).
-
-
 [![github license badge](https://img.shields.io/github/license/eWaterCycle/era5cli)](https://github.com/eWaterCycle/era5cli)
 [![rsd badge](https://img.shields.io/badge/RSD-era5cli-blue)](https://research-software-directory.org/software/era5cli)
 [![fair-software.eu](https://img.shields.io/badge/fair--software.eu-%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8B-yellow)](https://fair-software.eu)
@@ -27,6 +8,22 @@
 [![Test Coverage](https://codecov.io/gh/eWaterCycle/era5cli/branch/main/graph/badge.svg?token=qeZXgGASBK)](https://codecov.io/gh/eWaterCycle/era5cli)
 [![PyPI](https://badge.fury.io/py/era5cli.svg)](https://badge.fury.io/py/era5cli)
 
+<img align="right" width="150" alt="Logo" src="docs/assets/era5cli_logo_colors_border.png">
+
+> [!IMPORTANT]
+> The old Climate Data Store (CDS) has been shut down. All era5cli versions up to v1.4.2 will no longer work.
+> 
+> For more information see:
+> https://forum.ecmwf.int/t/goodbye-legacy-climate-data-store-hello-new-climate-data-store-cds/6380/14
+> 
+> To continue using era5cli, you will need to re-register at ECMWF and get a new API key,
+> and transition to era5cli version 2. This can be installed with:
+> `pip install era5cli==2.0.0` 
+
+> [!WARNING]
+> netCDF files from the new Climate Data Store Beta are not formatted the same as the
+> old CDS. Some variables might be missing.
+> See the open issue [here](https://github.com/eWaterCycle/era5cli/issues/165), as well as the [ECMWF discussion forum](https://forum.ecmwf.int/).  
 
 A command line interface to download ERA5 data from the [Copernicus Climate Data Store](<https://climate.copernicus.eu/>).
 

docs/CHANGELOG.md

@@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+# 2.0.0 - 2025-02-12
+
+Changes since v1.4.2:
+
+**Added:**
+
+ - support for Python 3.12, 3.13.
+
+**Changed:**
+
+ - The `splitmonths` argument now defaults to `True` for hourly requests. To not split requests by year, add `--splitmonths False`.
+ - The 'cads-api-client' used in the 2.0 beta versions is already deprecated, the backend now uses the 'cdsapi' again, which uses ["datapi"](https://github.com/ecmwf-projects/datapi).
+
+**Fixed:**
+
+ - Added support for the new climate data store.
+
+**Removed:**
+
+ - the deprecated `orography` variable. Use `geopotential` instead.
+ - the deprecated `--prelimbe` argument. This one has not been required anymore, as the back-extension is part of the normal dataset now.
+ - support for Python 3.8.
+
+**Dev changes:**
+
+ - The pre-commit hook has been removed. Pre-commit does not play well with hatch: it would need to be installed system-wide. No hatch-specific hooks are available.
+
+
+
 ## 2.0.0b2 - 2024-09-20
 
 **Fixed:**

docs/index.md

@@ -10,7 +10,7 @@ A command line interface to download ERA5 data from the [Copernicus Climate Data
 
     To continue using era5cli, you will need to re-register at ECMWF and get a new API key,
     and transition to the era5cli v2 beta. This can be installed with:
-    `pip install era5cli==2.0.0b2` 
+    `pip install era5cli>=2.0.0` 
 
 ???+ warning
     netCDF files from the new Climate Data Store Beta are not formatted the same as the

era5cli/__version__.py

@@ -26,4 +26,4 @@
     "Bart Schilperoort",
 )
 __email__ = "[email protected]"
-__version__ = "2.0.0b2"
+__version__ = "2.0.0"

era5cli/key_management.py

@@ -2,15 +2,16 @@
 import sys
 from pathlib import Path
 from typing import Tuple
-import cads_api_client
+import cdsapi
 from requests.exceptions import ConnectionError  # pylint: disable=redefined-builtin
 
 
 ERA5CLI_CONFIG_PATH = Path.home() / ".config" / "era5cli" / "cds_key.txt"
 CDSAPI_CONFIG_PATH = Path.home() / ".cdsapirc"
-DEFAULT_CDS_URL = "https://cds-beta.climate.copernicus.eu/api"
+DEFAULT_CDS_URL = "https://cds.climate.copernicus.eu/api"
 
-AUTH_ERR_MSG = "401 Client Error"
+AUTH_ERR_MSG = "401"
+NO_DATA_ERR_MSG = "There is no data matching your request"
 
 
 class InvalidRequestError(Exception):
@@ -34,9 +35,23 @@ def attempt_cds_login(url: str, key: str) -> True:
         InvalidRequestError: If the test request failed, likely due to changes in the
             CDS API's variable naming.
     """
-    client = cads_api_client.ApiClient(key, url)
+    client = cdsapi.Client(key=key, url=url, verify=True)
     try:
-        client.check_authentication()
+        # Check the URL
+        client.status()  # pragma: no cover
+
+        # Checks if the authentication works, without downloading data
+        client.retrieve(  # pragma: no cover
+            "reanalysis-era5-single-levels",
+            {
+                "variable": "2t",
+                "product_type": "reanalysis",
+                "date": "2012-12-01",
+                "time": "14:00",
+                "format": "netcdf",
+            },
+        )
+        return True
     except ConnectionError as err:
         raise ConnectionError(
             f"{os.linesep}Failed to connect to CDS. Please check your internet "
@@ -54,6 +69,11 @@ def attempt_cds_login(url: str, key: str) -> True:
                 f"{ERA5CLI_CONFIG_PATH.resolve()}{os.linesep}"
                 "Or redefine your configuration with 'era5cli config'"
             ) from err
+        if NO_DATA_ERR_MSG in str(err):
+            raise InvalidRequestError(
+                f"{os.linesep}Something changed in the CDS API. Please raise an issue "
+                "on https://www.github.com/eWaterCycle/era5cli"
+            ) from err
         raise err  # pragma: no cover
 
 
@@ -123,7 +143,7 @@ def load_era5cli_config() -> Tuple[str, str]:
                 "Old config detected. In the new CDS API only a key is required.\n"
                 "Please look at the new CDS website, and reconfigure your login in "
                 "era5cli\n"
-                "    https://cds-beta.climate.copernicus.eu/"
+                "    https://cds.climate.copernicus.eu/"
             )
             raise InvalidLoginError(msg)
 
@@ -148,7 +168,7 @@ def load_cdsapi_config() -> Tuple[str, str]:
             msg = (
                 "Your CDS API configuration file contains a UID entry/incorrect URL.\n"
                 "Please look at the new CDS website, and reconfigure your key:\n"
-                "    https://cds-beta.climate.copernicus.eu/"
+                "    https://cds.climate.copernicus.eu/"
             )
             raise InvalidLoginError(msg)
     return url, key

pyproject.toml

@@ -13,7 +13,7 @@ name = "era5cli"
 description = "A command line interface to download ERA5 data from the Copernicus Climate Data Store. https://climate.copernicus.eu/.."
 readme = "README.md"
 license = { text = "Apache Software License" }
-requires-python = ">=3.8, <3.12"
+requires-python = ">=3.9"
 authors = [
     {name = "Ronald van Haren"},
     {name = "Jaro Camphuijsen"},
@@ -51,14 +51,14 @@ classifiers = [
   "Operating System :: POSIX",
   "Programming Language :: Python",
   "Programming Language :: Python :: 3 :: Only",
-  "Programming Language :: Python :: 3.8",
   "Programming Language :: Python :: 3.9",
   "Programming Language :: Python :: 3.10",
   "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
 ]
 dependencies = [
-  "cdsapi>=0.7.1",
-  "cads-api-client==1.3.0",  # Pin because API is unstable: github.com/ecmwf-projects/cads-api-client/issues/81
+  "cdsapi>=0.7.4",
   "pathos",
   "PTable",
   "netCDF4"
@@ -111,7 +111,7 @@ serve = ["mkdocs serve",]
 
 [tool.black]
 line-length = 88
-target-version = ['py38', 'py39', 'py310', 'py311']
+target-version = ['py39', 'py310', 'py311', 'py312', 'py313']
 include = '\.pyi?$'
 
 [tool.isort]

tests/test_config.py

@@ -122,29 +122,41 @@ class TestAttemptCdsLogin:
     expected.
     """
 
-    @pytest.mark.xfail(reason="broken by new cads-client api")
     def test_status_fail(self):
-        with patch(
-            "cads_api_client.ApiClient.check_authentication",
-            side_effect=rex.ConnectionError,
-        ):
+        with patch("cdsapi.Client.status", side_effect=rex.ConnectionError):
             with pytest.raises(rex.ConnectionError, match="Failed to connect to CDS"):
-                key_management.attempt_cds_login(
-                    url="https://www.github.com/", key="def"
-                )
+                key_management.attempt_cds_login(url="test", key="abc:def")
 
     def test_connection_fail(self):
-        mp = patch(
-            "cads_api_client.ApiClient.check_authentication",
-            side_effect=Exception("401 Client Error"),
+        mp1 = patch("cdsapi.Client.status")
+        mp2 = patch(
+            "cdsapi.Client.retrieve",
+            side_effect=Exception("401"),
         )
-        with mp:
+        with mp1, mp2:
             with pytest.raises(
                 key_management.InvalidLoginError,
                 match="Authorization with the CDS served failed",
+            ):
+                key_management.attempt_cds_login(
+                    url="https://www.github.com/", key="abc:def"
+                )
+
+    def test_retrieve_fail(self):
+        mp1 = patch("cdsapi.Client.status")
+        mp2 = patch(
+            "cdsapi.Client.retrieve",
+            side_effect=Exception("There is no data matching your request"),
+        )
+        with mp1, mp2:
+            with pytest.raises(
+                key_management.InvalidRequestError,
+                match="Something changed in the CDS API",
             ):
                 key_management.attempt_cds_login(url="test", key="abc:def")
 
     def test_all_pass(self):
-        with patch("cads_api_client.ApiClient.check_authentication"):
-            key_management.attempt_cds_login(url="test", key="abc:def")
+        mp1 = patch("cdsapi.Client.status")
+        mp2 = patch("cdsapi.Client.retrieve")
+        with mp1, mp2:
+            assert key_management.attempt_cds_login(url="test", key="abc:def") is True