Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytest is broken on my computer #166

Closed
PhilipVinc opened this issue Jun 17, 2024 · 12 comments
Closed

Pytest is broken on my computer #166

PhilipVinc opened this issue Jun 17, 2024 · 12 comments

Comments

@PhilipVinc
Copy link
Collaborator

Since some recent changes in the testing infrastructure, I keep getting those errors

~/Dropbox/Ricerca/Codes/Python/plum pv/repr*
python-3.11.4pytest
================================================================================== test session starts ==================================================================================
platform darwin -- Python 3.11.4, pytest-8.2.1, pluggy-1.5.0
rootdir: /Users/filippo.vicentini/Dropbox/Ricerca/Codes/Python/plum
configfile: pyproject.toml
testpaths: tests/, plum, docs
plugins: cov-4.1.0
collected 417 items

tests/advanced/test_advanced.py ................                                                                                                                                  [  3%]
tests/advanced/test_annotated.py .                                                                                                                                                [  4%]
tests/advanced/test_cases.py ....                                                                                                                                                 [  5%]
tests/advanced/test_correctness.py ..........                                                                                                                                     [  7%]
tests/advanced/test_future_annotations.py .....                                                                                                                                   [  8%]
tests/advanced/test_precedence.py .                                                                                                                                               [  8%]
tests/advanced/test_return_type.py ....                                                                                                                                           [  9%]
tests/test_alias.py ....                                                                                                                                                          [ 10%]
tests/test_autoreload.py ..                                                                                                                                                       [ 11%]
tests/test_cache.py ....                                                                                                                                                          [ 12%]
tests/test_dispatcher.py ....                                                                                                                                                     [ 13%]
tests/test_function.py ..........................                                                                                                                                 [ 19%]
tests/test_init.py x...                                                                                                                                                           [ 20%]
tests/test_method.py ......                                                                                                                                                       [ 21%]
tests/test_parametric.py .....................                                                                                                                                    [ 26%]
tests/test_promotion.py ......                                                                                                                                                    [ 28%]
tests/test_repr.py ..                                                                                                                                                             [ 28%]
tests/test_resolver.py .........                                                                                                                                                  [ 30%]
tests/test_signature.py ..................                                                                                                                                        [ 35%]
tests/test_type.py .......................                                                                                                                                        [ 40%]
tests/test_util.py ........                                                                                                                                                       [ 42%]
tests/typechecked/test_overload.py .                                                                                                                                              [ 42%]
plum/parametric.py ................................                                                                                                                               [ 50%]
docs/autoreload.md ....                                                                                                                                                           [ 51%]
docs/basic_usage.md .......                                                                                                                                                       [ 53%]
docs/classes.md ............                                                                                                                                                      [ 56%]
docs/command_line.md .                                                                                                                                                            [ 56%]
docs/comparison.md .ss......ssssssss.....
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/main.py", line 285, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/main.py", line 339, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/logging.py", line 807, in pytest_runtestloop
INTERNALERROR>     return (yield)  # Run all the tests.
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/main.py", line 364, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/warnings.py", line 111, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/assertion/__init__.py", line 176, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/unittest.py", line 421, in pytest_runtest_protocol
INTERNALERROR>     res = yield
INTERNALERROR>           ^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/faulthandler.py", line 85, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/runner.py", line 116, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/runner.py", line 135, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/runner.py", line 243, in call_and_report
INTERNALERROR>     report: TestReport = ihook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/tmpdir.py", line 318, in pytest_runtest_makereport
INTERNALERROR>     rep = yield
INTERNALERROR>           ^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/skipping.py", line 269, in pytest_runtest_makereport
INTERNALERROR>     rep = yield
INTERNALERROR>           ^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/runner.py", line 368, in pytest_runtest_makereport
INTERNALERROR>     return TestReport.from_item_and_call(item, call)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/reports.py", line 363, in from_item_and_call
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>                ^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/sybil/integration/pytest.py", line 117, in repr_failure
INTERNALERROR>     return super().repr_failure(excinfo, style)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/nodes.py", line 482, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/nodes.py", line 462, in _repr_failure_py
INTERNALERROR>     return excinfo.getrepr(
INTERNALERROR>            ^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/_code/code.py", line 696, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/_code/code.py", line 1063, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo_)
INTERNALERROR>                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/_code/code.py", line 972, in repr_traceback
INTERNALERROR>     traceback = self.tbfilter(excinfo)
INTERNALERROR>                 ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/sybil/integration/pytest.py", line 97, in _traceback_filter
INTERNALERROR>     tb_entry = tb[1]
INTERNALERROR>                ~~^^^
INTERNALERROR>   File "/Users/filippo.vicentini/Documents/pythonenvs/plum-dev/python-3.11.4/lib/python3.11/site-packages/_pytest/_code/code.py", line 392, in __getitem__
INTERNALERROR>     return super().__getitem__(key)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> IndexError: list index out of range

Does anyone know why?

@nstarman
Copy link
Contributor

I just started getting lots of errors like

jax._src.source_info_util.JaxStackTraceBeforeTransformation: plum.resolver.MethodRedefinitionWarning: `Method(function_name='_promotion_rule', signature=Signature(typing.Any, typing.Any), return_type=typing.Any, impl=<function _promotion_rule at 0x7fe745291a20>)` overwrites the earlier definition `Method(function_name='_promotion_rule', signature=Signature(typing.Any, typing.Any), return_type=typing.Any, impl=<function _promotion_rule at 0x7fe745291a20>)`.

@wesselb
Copy link
Member

wesselb commented Jun 17, 2024

@PhilipVinc I'm experiencing the same issue. Haven't yet figured out what's wrong.

@wesselb
Copy link
Member

wesselb commented Jun 17, 2024

I think a doctest is the culprit:

$ pytest docs/comparison.md
<the above error>

@wesselb
Copy link
Member

wesselb commented Jun 17, 2024

For some reason, the below refuses to run as a doctest and now throws an error:

from plum import dispatch, parametric
from typing import Any, Optional, Tuple, Union

import numpy as np


class NDArrayMeta(type):
    def __instancecheck__(self, x):
        if self.concrete:
            shape, dtype = self.type_parameter
        else:
            shape, dtype = None, None
        return (
            isinstance(x, np.ndarray)
            and (shape is None or x.shape == shape)
            and (dtype is None or x.dtype == dtype)
        )


@parametric
class NDArray(np.ndarray, metaclass=NDArrayMeta):
    @classmethod
    @dispatch
    def __init_type_parameter__(
        cls,
        shape: Optional[Tuple[int, ...]],
        dtype: Optional[Any],
    ):
        """Validate the type parameter."""
        return shape, dtype

    @classmethod
    @dispatch
    def __le_type_parameter__(
        cls,
        left: Tuple[Optional[Tuple[int, ...]], Optional[Any]],
        right: Tuple[Optional[Tuple[int, ...]], Optional[Any]],
    ):
        """Define an order on type parameters. That is, check whether
        `left <= right` or not."""
        shape_left, dtype_left = left
        shape_right, dtype_right = right
        return (
            (shape_right is None or shape_left == shape_right)
            and (dtype_right is None or dtype_left == dtype_right)
        )


@dispatch
def f(x: np.ndarray):
    print("Any NP array!")


@dispatch
def f(x: NDArray[(2, 2), None]):
    print("A 2x2 array!")


@dispatch
def f(x: NDArray[None, int]):
    print("An int array!")


@dispatch
def f(x: NDArray[(2, 2), int]):
    print("A 2x2 int array!")


f(np.ones((3, 3)))       # Any NP array!
f(np.ones((3, 3), int))  # An int array!
f(np.ones((2, 2)))       # A 2x2 array!
f(np.ones((2, 2), int))  # A 2x2 int array!

@wesselb
Copy link
Member

wesselb commented Jun 17, 2024

Upgrading beartype seems to do the trick for me locally. @PhilipVinc, could you give that a go?

@PhilipVinc
Copy link
Collaborator Author

(Numpy 2.0 was released today and I guess that might cause some breakage)

@PhilipVinc
Copy link
Collaborator Author

yep

@wesselb
Copy link
Member

wesselb commented Jun 18, 2024

@nstarman Are you still experiencing issues?

@nstarman
Copy link
Contributor

nstarman commented Jun 19, 2024

Yes. I just opened GalacticDynamics/galax#357, which works on my un-upgraded plum/numpy/etc.
I'm seeing double registrations or re-registrations. It's hard to track down exactly where / what's occurring.

@nstarman
Copy link
Contributor

I went lower down the stack. I'm seeing the same thing in GalacticDynamics/quaxed#56

@wesselb
Copy link
Member

wesselb commented Jun 19, 2024

@nstarman I can confirm that I’m also seeing double registrations in some packages of mine. This seems to be a separate issue from the current thread, and revealed by the recently added warning. Do the registrations only pollute your output or actually break something? I will keep investigating on my end.

@nstarman
Copy link
Contributor

Let's move this do a different thread, but the issue seems to be that plum is trying to register the same function twice! E.g. I'm seeing this and both impl have the exact same memory address.

plum.resolver.MethodRedefinitionWarning: `Method(function_name='arange', signature=Signature(typing.Union[ArrayLike], typing.Optional[ArrayLike], typing.Optional[A, return_type=typing.Union[ArrayLike], impl=<function arange at 0x7feadc790af0>)` overwrites the earlier definition `Method(function_name='arange', signature=Signature(typing.Union[ArrayLike], typing.Optional[ArrayLike], typing.Optional[A, return_type=typing.Union[ArrayLike], impl=<function arange at 0x7feadc790af0>)`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants