Introduce @deprecated decorators for functions and enforce warnings as errors in tests #4757

RohitP2005 · 2025-01-14T19:55:31Z

Description

This pull request has implemented deprecation decorators as wrapper for a function's arguement and the fucntion itself and has added -W flag to ensure the deprecation warnings are treated as error

Related to: #2028

Type of change

New feature (non-breaking change that adds functionality)
Optimization (back-end change that speeds up the code)
Bug fix (non-breaking change which fixes an issue)
Deprecation (non-breaking change that deprecates functionality with warnings)

Key checklist:

No style issues: $ pre-commit run --all-files (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally)
All tests pass: $ python -m pytest -W error (or $ nox -s tests) (ensure warnings are treated as errors to catch deprecation notices)
The documentation builds: $ python -m pytest --doctest-plus src (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once using $ nox -s quick.

Further checks:

Code is commented, particularly in hard-to-understand areas, including deprecation warnings and helper functions.
Tests added for any new functionality or to prove that the deprecation warnings are effective.
Confirm that any borrowed code from Astropy is appropriately attributed according to its license.

RohitP2005 · 2025-01-14T19:55:59Z

Previous Discussions : #4745 (comment)

utils/exceptions.py

src/pybamm/simulation.py

src/pybamm/solvers/base_solver.py

kratman

I think one of the main reasons for doing this is to track the date of the deprecation warnings. This does not seem to be in these changes.

Realistically the warnings work as-is, but we want to automate the removal of them. Right now I use git-blame to remove old warnings every 6 months or so. The extra decorators and such are just additional overhead unless we use it for something better than the current warnings

kratman · 2025-01-14T20:08:11Z

Original issue request: "Replace existing deprecation warnings/errors with deprecation package so tests fail after a few releases."

RohitP2005 · 2025-01-14T20:17:41Z

Okay , I misunderstood that the deprecation package was not required and we needed a pythonic way.
Now i understand @kratman , Thankyou for the clarification .

I will just go ahead with the deprecation package with my next commit

Apologies for the confusion

RohitP2005 · 2025-01-16T20:06:17Z

@kratman, I've implemented the deprecation package for four of the existing deprecation warnings. Could you please guide me on any potential improvements?

Regarding renamed and deprecated arguments, it seems the deprecation package doesn't natively handle these cases. Do you have any suggestions on how to approach this? I considered writing custom decorators to manage them. Does that sound like a good plan?

codecov · 2025-01-16T20:18:17Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.71%. Comparing base (0d5af5c) to head (808d5cd).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #4757   +/-   ##
========================================
  Coverage    98.70%   98.71%           
========================================
  Files          304      304           
  Lines        23432    23452   +20     
========================================
+ Hits         23129    23151   +22     
+ Misses         303      301    -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

kratman · 2025-01-17T14:15:37Z

Does the package you use allow for tests to fail when the deprecation is old?

src/pybamm/simulation.py

RohitP2005 · 2025-01-20T03:16:18Z

Does the package you use allow for tests to fail when the deprecation is old?

No, the deprecation package does not natively support automatically failing tests when a deprecation is old

agriyakhetarpal

I think I agree with the general idea here, and the @deprecated decorator looks neat. If there's no way to make the deprecated functions fail directly, that's fine – we should proceed with your suggestion in the PR description:

has added -W flag to ensure the deprecation warnings are treated as an error

I don't see this change here, but yes, please feel free to configure pytest with this, as the deprecation warnings would then be caught in the tests if they use it. Then, you can fix any pending deprecation warnings unrelated to your changes that show up in the test suite. If there are a bunch of them and they are not straightforward to fix, it is also okay to filter them using the filterwarnings configuration option and open a new issue about removing them to work on later.

See also: https://docs.pytest.org/en/stable/how-to/capture-warnings.html#controlling-warnings

Regarding deprecating a specific argument in a function, yes, adding a custom decorator for this sounds great. If you think it will require a bit of experimentation, I'm happy to help review it in a follow-up PR (you may edit the PR description to mark this PR as related to #2028, but not fixing it). Limiting the scope of this PR sounds good to me to get it in.

Thanks!

agriyakhetarpal

Thanks, @RohitP2005! Could you please edit the PR title to make it more specific? As discussed, the follow-up task would be to address #4757 (review). I have a few more changes to suggest. We should also replace any other existing deprecation warnings with the decorators you added.

utils/__init__.py

utils/deprecation_Decorators.py

pyproject.toml

kratman · 2025-02-13T14:13:03Z

pyproject.toml

+    # convert internal warnings as errors
+    "error::DeprecationWarning:pybamm.*",
+    #ignore external DeprecationWarning


Have you confirmed that it works in this order? Since you say that they are errors, then right below ignore all deprecation warnings

Yes, it would be good to double-check this. Usually, the right order is to make PyBaMM-specific warnings errors, and then ignore any other warnings (such as UserWarnings that are expected). But at the same time, all warnings should start as errors.

@kratman
I suggeest we should order them in following way

filterwarnings = [ "error", # ignore external DeprecationWarning "ignore::DeprecationWarning", # convert internal warnings as errors "error::DeprecationWarning:pybamm.*", # ignore internal nbmake warnings 'ignore:unclosed \<socket.socket:ResourceWarning', 'ignore:unclosed event loop \<:ResourceWarning', "ignore::UserWarning", "ignore::RuntimeWarning", ]

"error": Treat all warnings as errors by default.

"ignore::DeprecationWarning": Ignore all DeprecationWarnings unless specified below.

Ignoring internal nbmake warnings: Ignore specific ResourceWarnings related to unclosed sockets and event loops.

"error::DeprecationWarning:pybamm.*": Treat PyBaMM-specific DeprecationWarnings as errors.

Ignoring other warnings: Ignore UserWarning and RuntimeWarning, which are expected but not critical.

This order ensures all warnings are caught as errors, while selectively ignoring or treating warnings related to PyBaMM and internal processes.

On second thought, one thing here is that we should also ensure that we aren't completely ignoring DeprecationWarnings added by projects. For example, we wouldn't want to use deprecated APIs by other libraries on which we depend. Doing so will make us vulnerable to things breaking, as we haven't pinned those libraries – that is, when they cut a new release wherein those deprecated APIs are removed, functionality from older PyBaMM versions won't be usable later in time, because we would not have seen those deprecation warnings in our tests and the newer versions of those dependencies would be fetched (unless we make an active effort to bump our minimum bounds for our dependencies to cap resolution for older PyBaMM versions, which we should).

So, having rethought this, my suggestion is to stick to https://learn.scientific-python.org/development/guides/pytest/#configuring-pytest as much as possible, ignore external DeprecationWarnings that cannot be fixed, and fix any other DeprecationWarnings (or ignore them and leave them as TODOs if they are hard to fix).

Co-authored-by: Eric G. Kratz <[email protected]>

agriyakhetarpal

Thanks, @RohitP2005! Could you please resolve the conflicts here? It looks close to merging with not many more changes required.

agriyakhetarpal · 2025-02-16T06:25:17Z

pyproject.toml

+    # convert internal warnings as errors
+    "error::DeprecationWarning:pybamm.*",
+    #ignore external DeprecationWarning


On second thought, one thing here is that we should also ensure that we aren't completely ignoring DeprecationWarnings added by projects. For example, we wouldn't want to use deprecated APIs by other libraries on which we depend. Doing so will make us vulnerable to things breaking, as we haven't pinned those libraries – that is, when they cut a new release wherein those deprecated APIs are removed, functionality from older PyBaMM versions won't be usable later in time, because we would not have seen those deprecation warnings in our tests and the newer versions of those dependencies would be fetched (unless we make an active effort to bump our minimum bounds for our dependencies to cap resolution for older PyBaMM versions, which we should).

So, having rethought this, my suggestion is to stick to https://learn.scientific-python.org/development/guides/pytest/#configuring-pytest as much as possible, ignore external DeprecationWarnings that cannot be fixed, and fix any other DeprecationWarnings (or ignore them and leave them as TODOs if they are hard to fix).

utils/__init__.py

utils/deprecation_Decorators.py

agriyakhetarpal · 2025-02-16T06:32:56Z

utils/deprecation_Decorators.py

+import functools
+import warnings
+
+
+def deprecate_arguments(deprecated_args, deprecated_in, removed_in, current_version):


Let's add type hints here while we can:

Suggested change

import functools

import warnings

def deprecate_arguments(deprecated_args, deprecated_in, removed_in, current_version):

from __future__ import annotations

import functools

import warnings

def deprecate_arguments(deprecated_args: dict[str], deprecated_in: str, removed_in: str, current_version: str):

agriyakhetarpal

Thanks, @RohitP2005! Could you please resolve the conflicts here? It looks close to merging with not many more changes required.

RohitP2005 · 2025-02-19T16:03:07Z

Thanks, @RohitP2005! Could you please resolve the conflicts here? It looks close to merging with not many more changes required.

Okay @agriyakhetarpal will address all those changes in next commit as requested.

pyproject.toml

agriyakhetarpal · 2025-02-20T11:34:21Z

pyproject.toml

@@ -216,6 +217,8 @@ required_plugins = [
 addopts = [
    "-nauto",
    "-vra",
+    "-ra",
+    "--showlocals",


I'm curious: is there a reason you think we should opt for this option? Do you feel that our tracebacks are limited?

pyproject.toml

src/pybamm/expression_tree/symbol.py

agriyakhetarpal · 2025-02-20T11:40:46Z

src/pybamm/parameters/parameter_values.py

+    @deprecate_arguments(
+        deprecated_args={
+            "electrode diffusivity": "Use 'particle diffusivity' instead.",
+            "1 + dlnf/dlnc": "Use 'Thermodynamic factor' instead.",
+        },
+        deprecated_in="24.1.0",
+        removed_in="25.3.0",
+        current_version="25.1.0",
+        msg="Update your parameter names to avoid future compatibility issues.",
+    )


Could we have multiple messages per argument? The current message here is not informative enough, and is rather a step back in comparison to the messages that currently exist already. That is, it would be nice if @deprecate_arguments() could take a dictionary, i.e., accept a key-value pair for an argument and its corresponding deprecation message. The current approach you've taken doesn't seem to accommodate this behaviour.

src/pybamm/solvers/base_solver.py

agriyakhetarpal · 2025-02-20T11:42:44Z

I've reviewed the changes again. Sorry if you were adding any other changes during my review!

Co-authored-by: Agriya Khetarpal <[email protected]>

RohitP2005 · 2025-02-22T18:18:13Z

Hey @kratman could u trigger the CIs for this one
@agriyakhetarpal I think this PR is good to go now.

RohitP2005 · 2025-02-22T18:20:05Z

@arjxn-py Thanks for initiating the CI's
Can anyone help me with the pkg_resources is deprecated as an API
I tried silencing it and still it pops up
Any possible fix for this

RohitP2005 requested review from martinjrobins, Saransh-cpp, agriyakhetarpal, kratman, arjxn-py, a team, rtimms, valentinsulzer, TomTranter, MarcBerliner, brosaplanella and DrSOKane as code owners January 14, 2025 19:55

kratman reviewed Jan 14, 2025

View reviewed changes

utils/exceptions.py Outdated Show resolved Hide resolved

kratman reviewed Jan 14, 2025

View reviewed changes

src/pybamm/simulation.py Outdated Show resolved Hide resolved

kratman reviewed Jan 14, 2025

View reviewed changes

src/pybamm/solvers/base_solver.py Outdated Show resolved Hide resolved

kratman requested changes Jan 14, 2025

View reviewed changes

Deprecation for deprecated fucntions

ff6a817

RohitP2005 force-pushed the issue#2028 branch from 5024850 to ff6a817 Compare January 16, 2025 19:55

style: pre-commit fixes

7f34a0b

prady0t suggested changes Jan 19, 2025

View reviewed changes

src/pybamm/simulation.py Outdated Show resolved Hide resolved

src/pybamm/simulation.py Outdated Show resolved Hide resolved

RohitP2005 and others added 2 commits February 2, 2025 19:08

Merge branch 'pybamm-team:develop' into issue#2028

3da7140

Removed previous deprecation warnings

3a15fe8

agriyakhetarpal requested changes Feb 2, 2025

View reviewed changes

Merge branch 'develop' into issue#2028

dba1881

RohitP2005 marked this pull request as ready for review February 13, 2025 13:53

agriyakhetarpal requested changes Feb 13, 2025

View reviewed changes

utils/__init__.py Outdated Show resolved Hide resolved

utils/deprecation_Decorators.py Outdated Show resolved Hide resolved

kratman reviewed Feb 13, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

kratman reviewed Feb 13, 2025

View reviewed changes

RohitP2005 changed the title ~~Deprecation decorators for functions~~ Introduce @deprecated decorators for functions and enforce warnings as errors in tests Feb 14, 2025

RohitP2005 and others added 2 commits February 14, 2025 11:36

Merge branch 'pybamm-team:develop' into issue#2028

c48ed38

Update pyproject.toml

b91e9ce

Co-authored-by: Eric G. Kratz <[email protected]>

agriyakhetarpal reviewed Feb 16, 2025

View reviewed changes

This comment was marked as duplicate.

Sign in to view

RohitP2005 and others added 7 commits February 20, 2025 10:46

moving deprecation decorator to uitl.py

c564c87

Merge branch 'develop' into issue#2028

52e610b

style: pre-commit fixes

a4025a1

Moved decorators to utils and corrected pytest config

33ff787

Added msg parameter in deprecation decorator

9d9c1bf

Applied decorators to necessary functions

139704c

Corrected imports for utils

808d5cd

agriyakhetarpal reviewed Feb 20, 2025

View reviewed changes

RohitP2005 and others added 4 commits February 20, 2025 22:16

Update pyproject.toml

28f5397

Co-authored-by: Agriya Khetarpal <[email protected]>

Update pyproject.toml

a703eca

Co-authored-by: Agriya Khetarpal <[email protected]>

removed old deprecation warningsdfasf

8784b35

deprecation Decorator accepts msg as a dictionary

7902c43

RohitP2005 requested review from agriyakhetarpal and kratman February 20, 2025 17:35

Merge branch 'develop' into issue#2028

eaf0994

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce @deprecated decorators for functions and enforce warnings as errors in tests #4757

Introduce @deprecated decorators for functions and enforce warnings as errors in tests #4757

RohitP2005 commented Jan 14, 2025 •

edited

Loading

RohitP2005 commented Jan 14, 2025

kratman left a comment

kratman commented Jan 14, 2025

RohitP2005 commented Jan 14, 2025 •

edited

Loading

RohitP2005 commented Jan 16, 2025

codecov bot commented Jan 16, 2025 •

edited

Loading

kratman commented Jan 17, 2025

RohitP2005 commented Jan 20, 2025

agriyakhetarpal left a comment

agriyakhetarpal left a comment

kratman Feb 13, 2025

agriyakhetarpal Feb 13, 2025

RohitP2005 Feb 14, 2025 •

edited

Loading

agriyakhetarpal Feb 16, 2025

agriyakhetarpal left a comment

agriyakhetarpal Feb 16, 2025

agriyakhetarpal Feb 16, 2025

agriyakhetarpal left a comment

This comment was marked as duplicate.

RohitP2005 commented Feb 19, 2025

agriyakhetarpal Feb 20, 2025

agriyakhetarpal Feb 20, 2025

agriyakhetarpal commented Feb 20, 2025

RohitP2005 commented Feb 22, 2025

RohitP2005 commented Feb 22, 2025 •

edited

Loading

Introduce @deprecated decorators for functions and enforce warnings as errors in tests #4757

Are you sure you want to change the base?

Introduce @deprecated decorators for functions and enforce warnings as errors in tests #4757

Conversation

RohitP2005 commented Jan 14, 2025 • edited Loading

Description

Type of change

Key checklist:

Further checks:

RohitP2005 commented Jan 14, 2025

kratman left a comment

Choose a reason for hiding this comment

kratman commented Jan 14, 2025

RohitP2005 commented Jan 14, 2025 • edited Loading

RohitP2005 commented Jan 16, 2025

codecov bot commented Jan 16, 2025 • edited Loading

Codecov Report

kratman commented Jan 17, 2025

RohitP2005 commented Jan 20, 2025

agriyakhetarpal left a comment

Choose a reason for hiding this comment

agriyakhetarpal left a comment

Choose a reason for hiding this comment

kratman Feb 13, 2025

Choose a reason for hiding this comment

agriyakhetarpal Feb 13, 2025

Choose a reason for hiding this comment

RohitP2005 Feb 14, 2025 • edited Loading

Choose a reason for hiding this comment

agriyakhetarpal Feb 16, 2025

Choose a reason for hiding this comment

agriyakhetarpal left a comment

Choose a reason for hiding this comment

agriyakhetarpal Feb 16, 2025

Choose a reason for hiding this comment

agriyakhetarpal Feb 16, 2025

Choose a reason for hiding this comment

agriyakhetarpal left a comment

Choose a reason for hiding this comment

This comment was marked as duplicate.

RohitP2005 commented Feb 19, 2025

agriyakhetarpal Feb 20, 2025

Choose a reason for hiding this comment

agriyakhetarpal Feb 20, 2025

Choose a reason for hiding this comment

agriyakhetarpal commented Feb 20, 2025

RohitP2005 commented Feb 22, 2025

RohitP2005 commented Feb 22, 2025 • edited Loading

RohitP2005 commented Jan 14, 2025 •

edited

Loading

RohitP2005 commented Jan 14, 2025 •

edited

Loading

codecov bot commented Jan 16, 2025 •

edited

Loading

RohitP2005 Feb 14, 2025 •

edited

Loading

RohitP2005 commented Feb 22, 2025 •

edited

Loading