Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Now overloads with ambiguous self are handled properly, refs #11347 #17239

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

flaeppe
Copy link

@flaeppe flaeppe commented May 13, 2024

Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/dtypes/dtypes.py:1173: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:519: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:534: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:575: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:590: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:1145: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:1191: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/missing.py:258: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/io/formats/format.py:1573: error: Item "Sequence[float]" of "ndarray[Any, Any] | Sequence[float]" has no attribute "round"  [union-attr]
+ pandas/io/formats/format.py:1584: error: No overload variant of "__getitem__" of "Sequence" matches argument type "ndarray[Any, dtype[bool_]]"  [call-overload]
+ pandas/io/formats/format.py:1584: note: Possible overload variants:
+ pandas/io/formats/format.py:1584: note:     def __getitem__(self, int, /) -> float
+ pandas/io/formats/format.py:1584: note:     def __getitem__(self, slice, /) -> Sequence[float]
+ pandas/io/formats/format.py:1586: error: No overload variant of "__getitem__" of "Sequence" matches argument type "ndarray[Any, dtype[bool_]]"  [call-overload]
+ pandas/io/formats/format.py:1586: note: Possible overload variants:
+ pandas/io/formats/format.py:1586: note:     def __getitem__(self, int, /) -> float
+ pandas/io/formats/format.py:1586: note:     def __getitem__(self, slice, /) -> Sequence[float]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/test/iostream_test.py:820: error: Unused "type: ignore" comment  [unused-ignore]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/test_timefuncs.py:1213: error: Expression is of type "Any", not "TimestampSeries"  [assert-type]
+ tests/test_series.py:1458: error: Expression is of type "Any", not "DataFrame"  [assert-type]
+ tests/test_series.py:1464: error: Expression is of type "Any", not "DataFrame"  [assert-type]
+ tests/test_series.py:3187: error: Expression is of type "Any", not "Series[type[object]]"  [assert-type]
+ tests/test_plotting.py:598: error: Expression is of type "Any", not "Series[Any]"  [assert-type]
+ tests/test_plotting.py:602: error: Expression is of type "Any", not "Axes"  [assert-type]
+ tests/test_indexes.py:360: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:395: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:403: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[float]]"  [assert-type]
+ tests/test_indexes.py:447: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:490: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:506: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:517: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[float]]"  [assert-type]
+ tests/test_indexes.py:555: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:610: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]

mkdocs (https://github.com/mkdocs/mkdocs)
+ mkdocs/tests/config/config_options_tests.py:1679: error: Expression is of type "Any", not "dict[str, dict[Any, Any]]"  [assert-type]

@sobolevn
Copy link
Member

@flaeppe can you please analyze the output?

Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my old PR, wow :)
Thank you!

Someone else has to review it, obviously.

@flaeppe
Copy link
Author

flaeppe commented May 14, 2024

Yes, it's just a rebase of your old PR.

I've only tried to check the pandas changes, but I'm suspecting that now getting the following changes is incorrect:

+ pandas/io/formats/format.py:1573: error: Item "Sequence[float]" of "ndarray[Any, Any] | Sequence[float]" has no attribute "round"  [union-attr]
+ pandas/io/formats/format.py:1584: error: No overload variant of "__getitem__" of "Sequence" matches argument type "ndarray[Any, dtype[bool_]]"  [call-overload]
+ pandas/io/formats/format.py:1584: note: Possible overload variants:
+ pandas/io/formats/format.py:1584: note:     def __getitem__(self, int, /) -> float
+ pandas/io/formats/format.py:1584: note:     def __getitem__(self, slice, /) -> Sequence[float]
+ pandas/io/formats/format.py:1586: error: No overload variant of "__getitem__" of "Sequence" matches argument type "ndarray[Any, dtype[bool_]]"  [call-overload]
+ pandas/io/formats/format.py:1586: note: Possible overload variants:
+ pandas/io/formats/format.py:1586: note:     def __getitem__(self, int, /) -> float
+ pandas/io/formats/format.py:1586: note:     def __getitem__(self, slice, /) -> Sequence[float]

It's within the following function and stems from this line: https://github.com/pandas-dev/pandas/blob/283a2dcb2f91db3452a9d2ee299632a109b224f4/pandas/io/formats/format.py#L1561

I haven't been able to find a reproducible case but essentially what is happening here is that there's a narrowing of a union, via a helper similar to something like this:

def helper(val: Any) -> np.ndarray: ...

percentiles: np.ndarray | Sequence[float]
percentiles = helper(percentiles)
reveal_type(percentiles)  # Revealed type is "np.ndarray | Sequence[float]"

There's a couple of @overloads and stuff in there, but that percentiles value is coerced to a np.ndarray type but it seems that previously it was picked up on but now the union is kept instead. Not sure if it's an issue caused by these changes or if the changes revealed a new subsequent issue.

Here's a related issue and comments: #2008 (comment)

@flaeppe
Copy link
Author

flaeppe commented May 14, 2024

Correct me if I'm wrong, but if I'm reading the comments in #2008 and the Always narrow types section in #16472 (comment) correctly I don't think issue mentioned in the above comment #17239 (comment) is caused by the changes here.

@Viicos
Copy link
Contributor

Viicos commented May 17, 2024

Haven't looked into the PR yet, but I'm wondering if this now complies with the recently added chapter in the spec (see python/typing#1667)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Overload ambiguity is ignored in self-annotated methods
3 participants