Keep `TypeVar` arguments when narrowing generic subclasses with `isinstance` and `issubclass`. #17099

tyralla · 2024-04-05T07:54:31Z

@finite-state-machine When I first read your issue, I thought only a special case was left unconsidered. However, I then realized that Mypy's narrowing of generic classes often tends to insert Any for type variables and so silently reduces type safety. I put some effort into handling all relevant cases I could think of (including variadic generics, which is new to me), but it is likely I missed some. Would you like to have a look at it?

…stance` and `issubclass`.

for more information, see https://pre-commit.ci

…ance for consistency with Python < 3.11

…generics

This reverts commit a94db4a.

TeamSpen210 · 2024-04-05T12:19:13Z

Went through some of these examples:

mitmproxy: True positive??. The problem is that a dict[K, V] | Iterable[tuple[K, V]] is being checked against dict, resulting in dict[K, V] | dict[tuple[K, V], Any]. I guess such a dict would be valid, but that's really not the intent of this code obviously. I think this needs type negation to be safely expressed.
jinja: Same as mitmproxy.
pandas: True positive. Sequence[date] | tuple[ndarray, ndarray, ndarray] with an instance check of tuple succeeds with both not just the tuple. Here they could swap to list I think?
bidict: Same as mitmproxy except with Mapping.
prefect: False positive, but looks like an existing join vs union issue? The inferred result for enumerate(list[str] | list[tuple[Hashable, str]]) is enumerate[Sequence[Hashable]].
antidote: A now-redundant cast() call.
mypy: A now-redundant cast() call.
aioredis: This is a true positive, the dict values should be constrained to only by ResponseError not any exception. This repo is archived though, should probably remove it?
urllib3: Same iterable/mapping overlap.
ibis: True positive, the return type is wrong I think.
werkzeug: Same iterable/dict issue, then a unpack-string error following on from the wrong inference.

Something does seem to need to be done about the mapping/iterable overlap, it seems a lot of projects have very similar code which will be basically untypable right now in a subtle way. Would a special case make sense, or maybe a rule about refusing to narrow if some of the typevars are underermined in that union element?

finite-state-machine · 2024-04-05T15:42:27Z

@tyralla and @TeamSpen210: thank you so much for your work on this!

While I'm reasonably handy when it comes to using mypy, I have very little knowledge of its internals; I don't think I'm the right person to look over a patch.

(I would like to get comfortable with mypy's internals and start fixing bugs rather than just reporting them, but I'm not really sure where to start.)

JelleZijlstra · 2024-04-05T15:53:55Z

(I would like to get comfortable with mypy's internals and start fixing bugs rather than just reporting them, but I'm not really sure where to start.)

I think few of us are really comfortable with mypy's internals. I would start simply by reading a PR like this, trying to understand the new code and reading related code, and that hopefully will eventually expand to a broader knowledge of how mypy works.

tyralla · 2024-04-05T22:19:13Z

Something does seem to need to be done about the mapping/iterable overlap, it seems a lot of projects have very similar code which will be basically untypable right now in a subtle way. Would a special case make sense, or maybe a rule about refusing to narrow if some of the typevars are underermined in that union element?

Thank you for this very comprehensive analysis, @TeamSpen210. Given the amount of Mypy primer output, I'm happy that we seem to face only one problematic case. (I agree that the false positive for prefect is likely related to the joins issue but not this PR.)

I applied Pyright, Pyre (both on their playgrounds with the respective default settings), and Mypy (including the proposed changes) to this simplified mapping/iterable example:

from typing import Iterable

def f(x: dict[str, str] | Iterable[bytes]) -> None:
    if isinstance(x, dict):
        reveal_type(x)

Pyright: Type of "x" is "dict[str, str] | dict[bytes, Unknown]"
Pyre: Revealed type [-1]: Revealed type for x is typing.Dict[str, str].
Mypy: Revealed type is "Union[builtins.dict[builtins.str, builtins.str], builtins.dict[Any, Any], builtins.dict[builtins.bytes, Any]]"

So, Pyright and (changed) Mypy behave identically, except for the unnecessary dict[Any, Any] returned by Mypy. I will check if this has to do with the proposed changes.

On the other hand, Pyre only narrows to dict[str, str]. At first sight, this is due to Pyre only transferring type variable arguments if such a transfer would be complete. At least the following example leaves this impression:

def f(x: Iterable[bytes]) -> None:
    if isinstance(x, dict):
        reveal_type(x)  # Revealed type [-1]: Revealed type for `x` is `typing.Dict[typing.Any, typing.Any]`. (Pyre)

Personally, I would favour the Pyright-like approach. Adding an option for switching between Pyright's and Pyre's behaviour could also be an option. I also checked the typing specs' narrowing section, but this topic is not covered. Do you think it should, @JelleZijlstra?

…ithout crashing pattern matching)

…pe (without crashing pattern matching)

for more information, see https://pre-commit.ci

…pe (without crashing pattern matching)

…rrowing revealed by Mypy primer and pysparsk

…in narrowing revealed by Mypy primer and pysparsk

tyralla · 2024-04-06T11:36:24Z

So, Pyright and (changed) Mypy behave identically, except for the unnecessary dict[Any, Any] returned by Mypy. I will check if this has to do with the proposed changes.

After a few modifications, this PR leads to a narrowing completely consistent with Pyright for the discussed example:

from typing import Iterable

def f(x: dict[str, str] | Iterable[bytes]) -> None:
    if isinstance(x, dict):
        reveal_type(x)

Pyright: Type of "x" is "dict[str, str] | dict[bytes, Unknown]"
Pyre: Revealed type [-1]: Revealed type for x is typing.Dict[str, str].
Mypy: Revealed type is "Union[builtins.dict[builtins.str, builtins.str], builtins.dict[builtins.bytes, Any]]"

The primer's response is nearly the same as the one analysed by @TeamSpen210. The differences between both primer outputs are due to different item orders in unions and the removal of unnecessary "any generics" like in the discussed example.

So, this PR seems ready for a more complete review.

tyralla · 2024-04-12T19:52:52Z

An additional thought. Wouldn't it be preferable to warn more strictly that type narrowing results in code that is not type-safe? Maybe the --disallow-any-generics option is a candidate for enabling warnings like this:

# mypy: disallow-any-generics

from typing import Iterable

def f(x: Iterable[str]) -> None:
    if isinstance(x, dict):   # E: Iterable[str] does not provide parameters for all type variables of generic type "dict"
        x["I must be string"] = "I could be number"

finite-state-machine · 2024-04-12T19:57:20Z

Sorry I've been quiet on this; I've had a nasty bug for the past 10 days or so.

Is there anything I can do to help this along?

tyralla · 2024-04-12T20:20:57Z

Thanks for offering help!

I think this is generally ready for a maintainer's review. Luckily, @hauntsaninja seems to have it already on his list. But if you have time and motivation, checking and eventually extending the new tests so that they cover as many potential flaws of this PR as possible would be nice. That would increase my confidence in not introducing a nasty 10-day bug in Mypy...

finite-state-machine · 2024-04-15T23:52:57Z

I suspect I've re-discovered this bug again, with the following code:

from __future__ import annotations
from typing import (
        Sequence,
        Tuple,
        Union,
        )

class Class1: pass
class Class2: pass
class Class3: pass

Class1Spec = Union[Class1, Tuple[Class2, Class3]]

def some_func(steps: Union[Class1Spec, Sequence[Class1Spec]]) -> None:

    if isinstance(steps, tuple):
        reveal_type(steps)
                # got: Tuple[Any, ...]
                # expected: Union[
                #       Tuple[Class2, Class3],
                #       Tuple[
                #           Class1Spec=Union[Class1, Tuple[Class2, Class3],
                #           ...
                #           ],
                #       ]
    else:
        reveal_type(steps)
                # as expected (only Tuple[Class2, Class3] is eliminated)

I've tried to verify this is fixed using @tyralla's branch, but the first reveal_type() at line 17 is giving:

Union[Tuple[Any, ...], Tuple[Union[Class1, Tuple[Class2, Class3]], ...]]

rather than

Union[Tuple[Class2, Class3], Tuple[Union[Class1, Tuple[Class2, Class3]], ...]]

Without the patch, it gives:

Tuple[Any, ...]

tyralla · 2024-04-16T06:00:14Z

Thank you for finding this limitation (does "re-discovered" mean it's already reported?). This is because tuples are represented by TupleType instead of Instance, and this PR only captures (unions of) Instance for the current type. A simpler repro for the current state of this PR:

[case testKeepTypeVarArgsWhenNarrowingGenericsToTupleType]
from typing import Sequence, Tuple, Union

class C: ...
x: Union[Tuple[C], Sequence[Tuple[C]]]
if isinstance(x, tuple):
    reveal_type(x)  # N: Revealed type is "Union[builtins.tuple[Any, ...], builtins.tuple[Tuple[__main__.C], ...]]"
else:
    reveal_type(x)  # N: Revealed type is "typing.Sequence[Tuple[__main__.C]]"
[builtins fixtures/tuple.pyi]

finite-state-machine · 2024-04-16T14:25:54Z

@tyralla I was thinking "rediscovered" as in "this is a special case of #17099". If it's not, and it doesn't make sense to tack the extra work onto this PR, I'll file a separate issue.

…ceAndTuples to testKeepTypeVarArgsWhenNarrowingGenericsInUnionsWithIsInstance

This reverts commit 116b70e.

tyralla · 2024-04-16T19:10:21Z

No, I thought about it and now believe it should be included in this PR. I did this in a simple manner without considering that one could narrow a tuple to a tuple subclass (this should not happen often and seems more challenging to implement; I left a code comment).

The test case testKeepTypeVarArgsWhenNarrowingTupleTypeToTuple checks this additional feature.

tyralla · 2024-04-16T19:51:10Z

The new primer output is practically identical to the last one.

…generics

github-actions · 2024-11-03T05:14:56Z

Diff from mypy_primer, showing the effect of this PR on open source code:

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/cli/cloud/__init__.py:192: error: Incompatible types in assignment (expression has type "Hashable", variable has type "Sequence[Hashable]")  [assignment]

jinja (https://github.com/pallets/jinja)
+ src/jinja2/filters.py:168: error: Incompatible types in assignment (expression has type "Union[dict_items[Any, Any], dict_items[str, Any], dict_items[Tuple[str, Any], Any]]", variable has type "Iterable[Tuple[str, Any]]")  [assignment]

cwltool (https://github.com/common-workflow-language/cwltool)
+ cwltool/process.py: note: In function "relocateOutputs":
+ cwltool/process.py:315:51: error: Argument 1 to "_collectDirEntries" has incompatible type "str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | Any | None"; expected "CWLObjectType | MutableSequence[CWLObjectType] | None"  [arg-type]
+ tests/test_examples.py: note: In function "test_factory_partial_scatter":
+ tests/test_examples.py:327:12: error: Value of type "str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | Any | None" is not indexable  [index]
+ tests/test_examples.py:327:12: error: Value of type "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" is not indexable  [index]
+ tests/test_examples.py:327:12: error: No overload variant of "__getitem__" of "MutableSequence" matches argument type "str"  [call-overload]
+ tests/test_examples.py:327:12: note: Possible overload variants:
+ tests/test_examples.py:327:12: note:     def __getitem__(self, int, /) -> CWLOutputType
+ tests/test_examples.py:327:12: note:     def __getitem__(self, slice[Any, Any, Any], /) -> MutableSequence[CWLOutputType]
+ tests/test_examples.py:327:26: error: Invalid index type "int" for "MutableMapping[str, CWLOutputType]"; expected type "str"  [index]
+ tests/test_examples.py:327:29: error: Invalid index type "str" for "str"; expected type "SupportsIndex | slice[Any, Any, Any]"  [index]
+ tests/test_examples.py:328:12: error: Value of type "str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | Any | None" is not indexable  [index]
+ tests/test_examples.py:328:26: error: Invalid index type "int" for "MutableMapping[str, CWLOutputType]"; expected type "str"  [index]
+ tests/test_examples.py:329:12: error: Value of type "str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | Any | None" is not indexable  [index]
+ tests/test_examples.py:329:12: error: Value of type "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" is not indexable  [index]
+ tests/test_examples.py:329:12: error: No overload variant of "__getitem__" of "MutableSequence" matches argument type "str"  [call-overload]
+ tests/test_examples.py:329:12: note: Possible overload variants:
+ tests/test_examples.py:329:12: note:     def __getitem__(self, int, /) -> CWLOutputType
+ tests/test_examples.py:329:12: note:     def __getitem__(self, slice[Any, Any, Any], /) -> MutableSequence[CWLOutputType]
+ tests/test_examples.py:329:26: error: Invalid index type "int" for "MutableMapping[str, CWLOutputType]"; expected type "str"  [index]
+ tests/test_examples.py:329:29: error: Invalid index type "str" for "str"; expected type "SupportsIndex | slice[Any, Any, Any]"  [index]
+ tests/test_examples.py: note: In function "test_factory_partial_output":
+ tests/test_examples.py:342:12: error: Value of type "str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | Any | None" is not indexable  [index]
+ tests/test_examples.py:342:12: error: No overload variant of "__getitem__" of "MutableSequence" matches argument type "str"  [call-overload]
+ tests/test_examples.py:342:12: note: Possible overload variants:
+ tests/test_examples.py:342:12: note:     def __getitem__(self, int, /) -> CWLOutputType
+ tests/test_examples.py:342:12: note:     def __getitem__(self, slice[Any, Any, Any], /) -> MutableSequence[CWLOutputType]
+ tests/test_examples.py:342:27: error: Invalid index type "str" for "str"; expected type "SupportsIndex | slice[Any, Any, Any]"  [index]
+ tests/test_examples.py: note: In function "test_scatter_output_filenames":
+ tests/test_examples.py:1639:21: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None"  [type-var]
+ tests/test_examples.py:1639:28: error: Value of type "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" is not indexable  [index]
+ tests/test_examples.py:1639:28: error: No overload variant of "__getitem__" of "MutableSequence" matches argument type "str"  [call-overload]
+ tests/test_examples.py:1639:28: note: Possible overload variants:
+ tests/test_examples.py:1639:28: note:     def __getitem__(self, int, /) -> CWLOutputType
+ tests/test_examples.py:1639:28: note:     def __getitem__(self, slice[Any, Any, Any], /) -> MutableSequence[CWLOutputType]
+ tests/test_examples.py:1639:36: error: Invalid index type "str" for "str"; expected type "SupportsIndex | slice[Any, Any, Any]"  [index]
+ tests/test_examples.py:1639:63: error: Item "None" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "__iter__" (not iterable)  [union-attr]
+ tests/test_examples.py:1639:63: error: Item "int" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "__iter__" (not iterable)  [union-attr]
+ tests/test_examples.py:1639:63: error: Item "float" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "__iter__" (not iterable)  [union-attr]
+ tests/test_examples.py:1642:13: error: Item "None" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1642:13: error: Item "int" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1642:13: error: Item "float" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1642:13: error: Item "MutableSequence[CWLOutputType]" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1642:13: error: Item "MutableMapping[str, CWLOutputType]" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1643:17: error: Item "None" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1643:17: error: Item "int" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1643:17: error: Item "float" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1643:17: error: Item "MutableSequence[CWLOutputType]" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1643:17: error: Item "MutableMapping[str, CWLOutputType]" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1644:17: error: Item "None" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1644:17: error: Item "int" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1644:17: error: Item "float" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1644:17: error: Item "MutableSequence[CWLOutputType]" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]
+ tests/test_examples.py:1644:17: error: Item "MutableMapping[str, CWLOutputType]" of "Any | str | int | float | MutableSequence[CWLOutputType] | MutableMapping[str, CWLOutputType] | None" has no attribute "endswith"  [union-attr]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/tests/extension/date/array.py:105: error: Item "date" of "date | Any" has no attribute "astype"  [union-attr]
+ pandas/tests/extension/date/array.py:106: error: Item "date" of "date | Any" has no attribute "astype"  [union-attr]
+ pandas/tests/extension/date/array.py:107: error: Item "date" of "date | Any" has no attribute "astype"  [union-attr]

aioredis (https://github.com/aio-libs/aioredis)
+ aioredis/connection.py:206: error: Incompatible return value type (got "Any | Exception", expected "ResponseError")  [return-value]

bidict (https://github.com/jab/bidict)
+ bidict/_iter.py: note: In function "iteritems":
+ bidict/_iter.py:25:9: error: Incompatible types in "yield from" (actual type "tuple[Any, Any] | tuple[tuple[KT, VT], Any]", expected type "tuple[KT, VT]")  [misc]

urllib3 (https://github.com/urllib3/urllib3)
+ src/urllib3/_collections.py:352: error: Incompatible types in assignment (expression has type "tuple[str, str]", variable has type "str")  [assignment]

mypy (https://github.com/python/mypy)
+ mypy/types.py:3680: error: Redundant cast to "List[Type]"  [redundant-cast]
+ mypy/types.py:3680: note: See https://mypy.rtfd.io/en/stable/_refs.html#code-redundant-cast for more info

streamlit (https://github.com/streamlit/streamlit)
+ lib/streamlit/elements/widgets/time_widgets.py: note: In function "_parse_date_value":
+ lib/streamlit/elements/widgets/time_widgets.py:98:25: error: List comprehension has incompatible type List[Union[date, Any, None]]; expected List[date]  [misc]

spark (https://github.com/apache/spark)
+ python/pyspark/ml/linalg/__init__.py:603: error: Incompatible types in assignment (expression has type "dict_items[Any, Any] | dict_items[float, Any] | dict_items[int, float] | dict_items[tuple[int, float], Any]", variable has type "bytes | tuple[int, float] | Iterable[float] | Iterable[tuple[int, float]] | dict[int, float]")  [assignment]
+ python/pyspark/ml/functions.py:293: error: Incompatible types in assignment (expression has type "list[Any]", target has type "ndarray[Any, Any]")  [assignment]
+ python/pyspark/mllib/linalg/__init__.py:661: error: Incompatible types in assignment (expression has type "dict_items[Any, Any] | dict_items[float, Any] | dict_items[int, float] | dict_items[tuple[int, float], Any]", variable has type "bytes | tuple[int, float] | Iterable[float] | Iterable[tuple[int, float]] | dict[int, float]")  [assignment]

packaging (https://github.com/pypa/packaging)
+ src/packaging/markers.py:160: error: Item "tuple[Variable | Value | Op, ...]" of "Any | tuple[Variable | Value, Op, Variable | Value] | Sequence[MarkerAtom]" has no attribute "serialize"  [union-attr]
+ src/packaging/markers.py:160: error: Item "Sequence[MarkerAtom]" of "Any | tuple[Variable | Value, Op, Variable | Value] | Sequence[MarkerAtom]" has no attribute "serialize"  [union-attr]
+ src/packaging/markers.py:218: error: Item "Sequence[MarkerList | MarkerAtom | str]" of "MarkerList | tuple[Variable | Value, Op, Variable | Value] | Any" has no attribute "value"  [union-attr]
+ src/packaging/markers.py:218: error: Item "tuple[Variable | Value | Op, ...]" of "MarkerList | tuple[Variable | Value, Op, Variable | Value] | Any" has no attribute "value"  [union-attr]
+ src/packaging/markers.py:220: error: Item "Sequence[MarkerList | MarkerAtom | str]" of "MarkerList | tuple[Variable | Value, Op, Variable | Value] | Any" has no attribute "value"  [union-attr]
+ src/packaging/markers.py:220: error: Item "tuple[Variable | Value | Op, ...]" of "MarkerList | tuple[Variable | Value, Op, Variable | Value] | Any" has no attribute "value"  [union-attr]
+ src/packaging/markers.py:221: error: Item "Sequence[MarkerList | MarkerAtom | str]" of "MarkerList | tuple[Variable | Value, Op, Variable | Value] | Any" has no attribute "value"  [union-attr]
+ src/packaging/markers.py:221: error: Item "tuple[Variable | Value | Op, ...]" of "MarkerList | tuple[Variable | Value, Op, Variable | Value] | Any" has no attribute "value"  [union-attr]
+ src/packaging/markers.py:225: error: Argument 2 to "_eval_op" has incompatible type "MarkerList | tuple[Variable | Value, Op, Variable | Value] | Any"; expected "Op"  [arg-type]

kopf (https://github.com/nolar/kopf)
+ kopf/_core/reactor/subhandling.py:70: error: Argument "id" to "generate_id" has incompatible type "Callable[[NamedArg(int, 'retry'), NamedArg(datetime, 'started'), NamedArg(timedelta, 'runtime'), NamedArg(Mapping[str, str], 'annotations'), NamedArg(Mapping[str, str], 'labels'), NamedArg(Body, 'body'), NamedArg(Meta, 'meta'), NamedArg(Spec, 'spec'), NamedArg(Status, 'status'), NamedArg(Resource, 'resource'), NamedArg(str | None, 'uid'), NamedArg(str | None, 'name'), NamedArg(str | None, 'namespace'), NamedArg(Patch, 'patch'), NamedArg(str, 'reason'), NamedArg(Diff, 'diff'), NamedArg(BodyEssence | Any | None, 'old'), NamedArg(BodyEssence | Any | None, 'new'), NamedArg(Logger | LoggerAdapter[Any], 'logger'), NamedArg(Any, 'memo'), DefaultNamedArg(Any, 'param'), KwArg(Any)], object | Coroutine[None, None, object | None] | None]"; expected "str | None"  [arg-type]

scrapy (https://github.com/scrapy/scrapy)
+ scrapy/utils/datatypes.py:88: error: Incompatible types in assignment (expression has type "ItemsView[tuple[str, Any], Any] | Iterable[tuple[str, Any]]", variable has type "Mapping[str, Any] | Iterable[tuple[str, Any]]")  [assignment]
+ scrapy/utils/datatypes.py:88: error: Incompatible types in assignment (expression has type "ItemsView[tuple[bytes, Any], Any] | Iterable[tuple[bytes, Any]]", variable has type "Mapping[bytes, Any] | Iterable[tuple[bytes, Any]]")  [assignment]
+ scrapy/utils/datatypes.py:89: error: Unpacking a string is disallowed  [misc]
+ scrapy/utils/datatypes.py:89: error: Value of type variable "AnyStr" of "normkey" of "CaselessDict" cannot be "int | bytes"  [type-var]
+ scrapy/http/headers.py:37: error: Incompatible types in assignment (expression has type "ItemsView[tuple[str, Any], Any] | Iterable[tuple[str, Any]]", variable has type "Mapping[str, Any] | Iterable[tuple[str, Any]]")  [assignment]
+ scrapy/http/headers.py:37: error: Incompatible types in assignment (expression has type "ItemsView[tuple[bytes, Any], Any] | Iterable[tuple[bytes, Any]]", variable has type "Mapping[bytes, Any] | Iterable[tuple[bytes, Any]]")  [assignment]
+ scrapy/http/headers.py:39: error: Unpacking a string is disallowed  [misc]
+ scrapy/http/headers.py:40: error: Value of type variable "AnyStr" of "normkey" of "Headers" cannot be "int | bytes"  [type-var]

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ pymongo/helpers_shared.py:140: error: Argument 1 to "append" of "list" has incompatible type "tuple[str, int | str | Mapping[str, Any]]"; expected "tuple[str, int]"  [arg-type]
+ pymongo/synchronous/cursor.py:683: error: Argument 1 to "dict" has incompatible type "list[str | tuple[str, int | str | Mapping[str, Any]]] | list[Any] | tuple[str | tuple[str, int | str | Mapping[str, Any]], ...] | tuple[Any, ...]"; expected "Iterable[tuple[str | Any, Any]]"  [arg-type]
+ pymongo/synchronous/cursor.py:705: error: Argument 1 to "dict" has incompatible type "list[str | tuple[str, int | str | Mapping[str, Any]]] | list[Any] | tuple[str | tuple[str, int | str | Mapping[str, Any]], ...] | tuple[Any, ...]"; expected "Iterable[tuple[str | Any, Any]]"  [arg-type]
+ pymongo/asynchronous/cursor.py:685: error: Argument 1 to "dict" has incompatible type "list[str | tuple[str, int | str | Mapping[str, Any]]] | list[Any] | tuple[str | tuple[str, int | str | Mapping[str, Any]], ...] | tuple[Any, ...]"; expected "Iterable[tuple[str | Any, Any]]"  [arg-type]
+ pymongo/asynchronous/cursor.py:707: error: Argument 1 to "dict" has incompatible type "list[str | tuple[str, int | str | Mapping[str, Any]]] | list[Any] | tuple[str | tuple[str, int | str | Mapping[str, Any]], ...] | tuple[Any, ...]"; expected "Iterable[tuple[str | Any, Any]]"  [arg-type]

jax (https://github.com/google/jax)
+ jax/_src/interpreters/xla.py:73: error: Argument 1 has incompatible type "Callable[[Sequence[int | Any] | tuple[Sequence[int | Any] | None, ...] | None], Any]"; expected "Callable[[int | Any | Sequence[int | Any] | None], Any]"  [arg-type]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/cookiejar.py:237:23: error: Incompatible types in assignment (expression has type "ItemsView[str, str | BaseCookie[str] | Morsel[Any]] | ItemsView[tuple[str, str | BaseCookie[str] | Morsel[Any]], Any] | ItemsView[Any, Any] | dict_items[str, Morsel[str]]", variable has type "Mapping[str, str | BaseCookie[str] | Morsel[Any]] | Iterable[tuple[str, str | BaseCookie[str] | Morsel[Any]]] | BaseCookie[str]")  [assignment]
+ aiohttp/cookiejar.py:239:9: error: Unpacking a string is disallowed  [misc]
+ aiohttp/client_reqrep.py:376:23: error: Incompatible types in assignment (expression has type "ItemsView[str | istr, str] | ItemsView[str | istr, Any] | dict_items[tuple[str | istr, str], Any]", variable has type "Mapping[str, str] | Mapping[istr, str] | CIMultiDict[str] | CIMultiDictProxy[str] | Iterable[tuple[str | istr, str]] | None")  [assignment]
+ aiohttp/client_reqrep.py:424:19: error: Invalid index type "str | Any | tuple[str, str | BaseCookie[str] | Morsel[Any]]" for "SimpleCookie"; expected type "str"  [index]
+ aiohttp/client_reqrep.py:424:19: note: See https://mypy.rtfd.io/en/stable/_refs.html#code-index for more info
+ aiohttp/client_reqrep.py:426:19: error: Invalid index type "str | Any | tuple[str, str | BaseCookie[str] | Morsel[Any]]" for "SimpleCookie"; expected type "str"  [index]
+ aiohttp/client_reqrep.py:426:19: note: Error code "index" not covered by "type: ignore" comment
- aiohttp/web.py:328:25: error: Argument 2 to "TCPSite" has incompatible type "str | memoryview[Any]"; expected "str | None"  [arg-type]
+ aiohttp/web.py:328:25: error: Argument 2 to "TCPSite" has incompatible type "str | memoryview[str]"; expected "str | None"  [arg-type]

ibis (https://github.com/ibis-project/ibis)
+ ibis/util.py:133: error: Incompatible return value type (got "tuple[Any, ...] | tuple[V, ...]", expected "tuple[V]")  [return-value]

core (https://github.com/home-assistant/core)
+ homeassistant/auth/permissions/merge.py:63: error: Incompatible types in assignment (expression has type "Mapping[str, Mapping[str, Mapping[str, bool] | bool | None] | bool | None] | bool | None", target has type "Mapping[str, bool] | bool | None")  [assignment]
+ homeassistant/auth/permissions/util.py:113: error: Unused "type: ignore" comment  [unused-ignore]
+ homeassistant/auth/permissions/util.py:113: error: Incompatible return value type (got "Mapping[str, bool] | bool | Any | None", expected "bool")  [return-value]
+ homeassistant/auth/permissions/util.py:113: note: Error code "return-value" not covered by "type: ignore" comment
+ homeassistant/components/linear_garage_door/config_flow.py:117: error: Incompatible types in assignment (expression has type "list[Collection[str]]", variable has type "list[dict[str, str]]")  [assignment]

mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ mitmproxy/http.py:593: error: Generator has incompatible item type "tuple[None, bytes]"; expected "tuple[bytes, bytes]"  [misc]
+ mitmproxy/http.py:594: error: Argument 1 to "always_bytes" has incompatible type "str | bytes | Any | tuple[bytes, bytes]"; expected "None"  [arg-type]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures/structures.py:33: error: Incompatible types in "yield from" (actual type "Union[tuple[K, Union[V, Collection[V]]], tuple[tuple[K, V], Any]]", expected type "tuple[K, V]")  [misc]
+ src/werkzeug/datastructures/structures.py:38: error: Incompatible types in "yield" (actual type "tuple[Union[K, tuple[K, V]], Union[Any, V]]", expected type "tuple[K, V]")  [misc]
+ src/werkzeug/datastructures/headers.py:511: error: Argument 1 to "setlist" of "Headers" has incompatible type "Union[str, Any, tuple[str, Any]]"; expected "str"  [arg-type]
+ src/werkzeug/datastructures/headers.py:511: error: Argument 1 to "getlist" of "Headers" has incompatible type "Union[str, Any, tuple[str, Any]]"; expected "str"  [arg-type]
+ src/werkzeug/datastructures/headers.py:511: error: Argument 1 to "getlist" of "MultiDict" has incompatible type "Union[str, Any, tuple[str, Any]]"; expected "str"  [arg-type]
+ src/werkzeug/datastructures/headers.py:511: error: Argument 1 to "getlist" of "MultiDict" has incompatible type "Union[str, Any, tuple[str, Any]]"; expected "tuple[str, Any]"  [arg-type]
+ src/werkzeug/datastructures/headers.py:517: error: Argument 1 to "setlist" of "Headers" has incompatible type "Union[str, Any, tuple[str, Any]]"; expected "str"  [arg-type]
+ src/werkzeug/datastructures/headers.py:519: error: Argument 1 to "set" of "Headers" has incompatible type "Union[str, Any, tuple[str, Any]]"; expected "str"  [arg-type]
+ src/werkzeug/middleware/shared_data.py:121: error: Incompatible types in assignment (expression has type "Union[ItemsView[str, Union[str, tuple[str, str]]], ItemsView[tuple[str, Union[str, tuple[str, str]]], Any]]", variable has type "Union[Mapping[str, Union[str, tuple[str, str]]], Iterable[tuple[str, Union[str, tuple[str, str]]]]]")  [assignment]
+ src/werkzeug/middleware/shared_data.py:123: error: Unpacking a string is disallowed  [misc]

antidote (https://github.com/Finistere/antidote)
+ src/antidote/lib/interface_ext/_interface.py:531: error: Unused "type: ignore" comment  [unused-ignore]

tyralla and others added 4 commits April 5, 2024 01:34

Keep TypeVar arguments when narrowing generic subclasses with `isin…

84e3837

…stance` and `issubclass`.

[pre-commit.ci] auto fixes from pre-commit.com hooks

dbcfa1c

for more information, see https://pre-commit.ci

Remove a cast that is now redundant.

098e066

fix: Remove a cast that is now redundant.

892c74f

This comment has been minimized.

Sign in to view

tyralla added 5 commits April 5, 2024 10:43

Use Unpack in testKeepTypeVarTupleArgsWhenNarrowingGenericsWithIsInst…

c24cc53

…ance for consistency with Python < 3.11

fix "local variable 'pos2' referenced before assignment"

d013673

Merge branch 'master' into feature/keep_type_var_args_when_narrowing_…

fe51e85

…generics

restart job

a94db4a

Revert "restart job"

a700294

This reverts commit a94db4a.

This comment has been minimized.

Sign in to view

tyralla and others added 5 commits April 6, 2024 01:08

remove empty line

73da8b6

avoid unnecessary union members when proposed type is current type (w…

325f190

…ithout crashing pattern matching)

fix: avoid unnecessary union members when proposed type is current ty…

bb63532

…pe (without crashing pattern matching)

[pre-commit.ci] auto fixes from pre-commit.com hooks

18bae83

for more information, see https://pre-commit.ci

fix: avoid unnecessary union members when proposed type is current ty…

afa5450

…pe (without crashing pattern matching)

This comment has been minimized.

Sign in to view

apply one more union flattening to avoid (good or bad?) changes in na…

9769638

…rrowing revealed by Mypy primer and pysparsk

This comment has been minimized.

Sign in to view

fix: apply one more union flattening to avoid (good or bad?) changes …

6741175

…in narrowing revealed by Mypy primer and pysparsk

This comment has been minimized.

Sign in to view

hauntsaninja added the upnext label Apr 8, 2024

tyralla added 4 commits April 16, 2024 20:07

Change test name testKeepTypeVarArgsWhenNarrowingGenericsWithIsInstan…

463c7d4

…ceAndTuples to testKeepTypeVarArgsWhenNarrowingGenericsInUnionsWithIsInstance

special handling of TupleType

6195de1

restart Mypy primer

116b70e

Revert "restart Mypy primer"

ac4b9dc

This reverts commit 116b70e.

This comment has been minimized.

Sign in to view

Merge branch 'master' into feature/keep_type_var_args_when_narrowing_…

557bfe2

…generics

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Keep `TypeVar` arguments when narrowing generic subclasses with `isinstance` and `issubclass`. #17099

Keep `TypeVar` arguments when narrowing generic subclasses with `isinstance` and `issubclass`. #17099

tyralla commented Apr 5, 2024

This comment has been minimized.

This comment has been minimized.

TeamSpen210 commented Apr 5, 2024

finite-state-machine commented Apr 5, 2024 •

edited

Loading

JelleZijlstra commented Apr 5, 2024

tyralla commented Apr 5, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

tyralla commented Apr 6, 2024

tyralla commented Apr 12, 2024

finite-state-machine commented Apr 12, 2024

tyralla commented Apr 12, 2024

finite-state-machine commented Apr 15, 2024

tyralla commented Apr 16, 2024 •

edited

Loading

finite-state-machine commented Apr 16, 2024

tyralla commented Apr 16, 2024

This comment has been minimized.

tyralla commented Apr 16, 2024

github-actions bot commented Nov 3, 2024

Keep TypeVar arguments when narrowing generic subclasses with isinstance and issubclass. #17099

Are you sure you want to change the base?

Keep TypeVar arguments when narrowing generic subclasses with isinstance and issubclass. #17099

Conversation

tyralla commented Apr 5, 2024

This comment has been minimized.

This comment has been minimized.

TeamSpen210 commented Apr 5, 2024

finite-state-machine commented Apr 5, 2024 • edited Loading

JelleZijlstra commented Apr 5, 2024

tyralla commented Apr 5, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

tyralla commented Apr 6, 2024

tyralla commented Apr 12, 2024

finite-state-machine commented Apr 12, 2024

tyralla commented Apr 12, 2024

finite-state-machine commented Apr 15, 2024

tyralla commented Apr 16, 2024 • edited Loading

finite-state-machine commented Apr 16, 2024

tyralla commented Apr 16, 2024

This comment has been minimized.

tyralla commented Apr 16, 2024

github-actions bot commented Nov 3, 2024

Keep `TypeVar` arguments when narrowing generic subclasses with `isinstance` and `issubclass`. #17099

Keep `TypeVar` arguments when narrowing generic subclasses with `isinstance` and `issubclass`. #17099

finite-state-machine commented Apr 5, 2024 •

edited

Loading

tyralla commented Apr 16, 2024 •

edited

Loading