Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do not consider bare TypeVar not overlapping with None for reachability analysis #18138

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Do not consider bare TypeVar not overlapping with None for reachability analysis #18138

wants to merge 4 commits into from
+115 −42

Conversation

sterliakov
Copy link
Contributor

Fixes #18126.

Simply allowing such intersection was insufficient: existing binder logic widened the type to T | None after the is None check.

This PR extends the binder logic to prevent constructing a union type when all conditional branches are reachable and contain no assignments: checking if isinstance(something, Something) does not change the type of something after the end of the if block.

@sterliakov sterliakov marked this pull request as draft November 10, 2024 18:04
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@sterliakov
Copy link
Contributor Author

sterliakov commented Nov 10, 2024 •
edited
Loading

CI failed with the Windows problem I faced multiple times before, the rest is green.

Primer diff is amazing this time! Every diff line seems to be an improvement.

  • jinja2: unrelated meet-vs-join problem. Minimal repro (playground): 
    class A: pass
class B: pass

foos: tuple[A, ...] | tuple[B, ...]

for _, foo in enumerate(foos):
    reveal_type(foo)  # N: Revealed type is "builtins.object"
  • prefect: the error is correct. I don't know why it wasn't emitted previously, though: there's assignment of different types to an unannotated variable in if/else branches.
  • sphinx/search/__init__.py: bug in docutils-stubs (children is declared in Node, but not in stubs), and mypy is correct now.
  • sphinx/ext/graphviz.py: correct! This attribute isn't declared on any of the classes. Still don't know how it relates to this change, though...
  • schema_salad: correct! self.name is Any | None and is passed to a function expecting str.
  • kopf: another unrelated bug:( Minimal repro (playground with sync/async comparison): 
    from typing import Any, Callable, TypeVar

T = TypeVar("T")

async def call(fn: Callable[[], T]) -> T:
    return fn()

make_x: Any

async def fn() -> None:
    x: Any | None = None
    x = await call(make_x)
    reveal_type(x)  # N: Revealed type is "Union[Any, None]"
  • pytest: correct. There's really no error.
  • paroxython: correct, there's a narrowing not supported by mypy
  • pandera: fine, undeclared attributes
  • cki-lib: correct! It boils down to the following: 
    x: Any
if x is None:
    ...
do_something_with_not_none(x)
  • pygithub: correct! That is really a type error.
  • discord: all correct! Especially nice to get rid of Item "object" of the upper bound "Cog | None" - that was strange.
  • steam: both correct! Second one suffered from needless type widening after isinstance
  • ibis: all correct!

@sterliakov sterliakov marked this pull request as ready for review November 10, 2024 20:49
@github-actions GitHub Actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

jinja (https://github.com/pallets/jinja)
+ src/jinja2/lexer.py:761: error: "object" not callable  [operator]
+ src/jinja2/lexer.py:781: error: Incompatible types in "yield" (actual type "Tuple[int, object, str]", expected type "Tuple[int, str, str]")  [misc]

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/states.py:227: error: Incompatible types in assignment (expression has type "BaseException", variable has type "ResultRecord[Any]")  [assignment]

sphinx (https://github.com/sphinx-doc/sphinx)
+ sphinx/search/__init__.py:628:18: error: "Node" has no attribute "children"  [attr-defined]
+ sphinx/ext/graphviz.py: note: In function "render_dot":
+ sphinx/ext/graphviz.py:307:9: error: Item "StandaloneHTMLBuilder" of "StandaloneHTMLBuilder | LaTeXBuilder | TexinfoBuilder" has no attribute "_graphviz_warned_dot"  [union-attr]
+ sphinx/ext/graphviz.py:307:9: error: Item "LaTeXBuilder" of "StandaloneHTMLBuilder | LaTeXBuilder | TexinfoBuilder" has no attribute "_graphviz_warned_dot"  [union-attr]
+ sphinx/ext/graphviz.py:307:9: error: Item "TexinfoBuilder" of "StandaloneHTMLBuilder | LaTeXBuilder | TexinfoBuilder" has no attribute "_graphviz_warned_dot"  [union-attr]

schema_salad (https://github.com/common-workflow-language/schema_salad)
+ schema_salad/metaschema.py: note: In member "save" of class "EnumSchema":
+ schema_salad/metaschema.py:1882:49: error: Argument 2 to "save_relative_uri" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:1886:49: error: Argument "base_url" to "save" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py: note: In member "save" of class "SaladEnumSchema":
+ schema_salad/metaschema.py:5378:51: error: Argument "base_url" to "save" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5381:49: error: Argument 2 to "save_relative_uri" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5385:49: error: Argument "base_url" to "save" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5389:47: error: Argument "base_url" to "save" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5392:51: error: Argument 2 to "save_relative_uri" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5395:50: error: Argument 2 to "save_relative_uri" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5398:50: error: Argument 2 to "save_relative_uri" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5404:26: error: Argument "base_url" to "save" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5411:26: error: Argument "base_url" to "save" has incompatible type "Any | None"; expected "str"  [arg-type]
+ schema_salad/metaschema.py:5415:49: error: Argument 2 to "save_relative_uri" has incompatible type "Any | None"; expected "str"  [arg-type]

kopf (https://github.com/nolar/kopf)
+ kopf/_kits/webhooks.py:539: error: Item "None" of "Any | None" has no attribute "public_url"  [union-attr]

pytest (https://github.com/pytest-dev/pytest)
- src/_pytest/_code/code.py:1071: error: Argument 1 to "from_exception" of "ExceptionInfo" has incompatible type "Optional[BaseException]"; expected "BaseException"  [arg-type]
- src/_pytest/_code/code.py:1071: error: Item "None" of "Optional[BaseException]" has no attribute "__traceback__"  [union-attr]
- src/_pytest/_code/code.py:1077: error: Argument 1 to "from_exception" of "ExceptionInfo" has incompatible type "Optional[BaseException]"; expected "BaseException"  [arg-type]
- src/_pytest/_code/code.py:1077: error: Item "None" of "Optional[BaseException]" has no attribute "__traceback__"  [union-attr]

paroxython (https://github.com/laowantong/paroxython)
+ paroxython/flatten_ast.py:401: error: "AST" has no attribute "lineno"  [attr-defined]

pandera (https://github.com/pandera-dev/pandera)
- pandera/api/dataframe/model.py:72: error: Unused "type: ignore" comment  [unused-ignore]
- pandera/api/dataframe/model.py:77: error: Unused "type: ignore" comment  [unused-ignore]

cki-lib (https://gitlab.com/cki-project/cki-lib)
- cki_lib/s3bucket.py:57: error: Item "None" of "Any | None" has no attribute "get_credentials"  [union-attr]

PyGithub (https://github.com/PyGithub/PyGithub)
- github/GithubObject.py:315: error: Unused "type: ignore" comment  [unused-ignore]

discord.py (https://github.com/Rapptz/discord.py)
- discord/ext/commands/core.py:590: error: Unused "type: ignore" comment  [unused-ignore]
- discord/ext/commands/core.py:645: error: Item "object" of the upper bound "Cog | None" of type variable "CogT" has no attribute "cog_command_error"  [union-attr]
- discord/ext/commands/core.py:884: error: Item "object" of the upper bound "Cog | None" of type variable "CogT" has no attribute "cog_before_invoke"  [union-attr]
- discord/ext/commands/core.py:904: error: Item "object" of the upper bound "Cog | None" of type variable "CogT" has no attribute "cog_after_invoke"  [union-attr]
- discord/ext/commands/core.py:1148: error: "type[CogT]" has no attribute "__cog_name__"  [attr-defined]
- discord/ext/commands/core.py:1277: error: Item "object" of the upper bound "Cog | None" of type variable "CogT" has no attribute "cog_check"  [union-attr]

steam.py (https://github.com/Gobot1234/steam.py)
- steam/app.py:130: error: Argument "game_extra_info" to "CMsgClientGamesPlayedGamePlayed" has incompatible type "NameT"; expected "str"  [arg-type]
- steam/ext/commands/commands.py:923: error: Incompatible return value type (got "Command[Any, Any, Any] | MaybeCommandT", expected "MaybeBool | MaybeCommandT")  [return-value]

ibis (https://github.com/ibis-project/ibis)
+ ibis/util.py:137: error: Incompatible return value type (got "tuple[()]", expected "tuple[V]")  [return-value]
- ibis/selectors.py:708: error: Argument 1 to "ColumnIndex" has incompatible type "int | slice[Any, Any, Any] | Iterable[int | str]"; expected "str | int | Slice | tuple[int | str, ...]"  [arg-type]
+ ibis/selectors.py:708: error: Argument 1 to "ColumnIndex" has incompatible type "str | int | slice[Any, Any, Any] | Iterable[int | str]"; expected "str | int | Slice | tuple[int | str, ...]"  [arg-type]
- ibis/expr/types/relations.py:2629: error: Argument 3 to "DropNull" has incompatible type "Sequence[str] | None"; expected "tuple[Value[Any, Columnar], ...] | None"  [arg-type]
+ ibis/expr/types/relations.py:2629: error: Argument 3 to "DropNull" has incompatible type "Sequence[str] | str | None"; expected "tuple[Value[Any, Columnar], ...] | None"  [arg-type]
+ ibis/backends/sqlite/__init__.py:497: error: Item "None" of "Any | Any | Any | Any | Any | None" has no attribute "schema"  [union-attr]
- ibis/backends/duckdb/__init__.py:532: error: Argument 1 to "read_parquet" of "Backend" has incompatible type "str | Any | Path | list[Any] | tuple[Any, ...]"; expected "str | Iterable[str]"  [arg-type]
+ ibis/backends/duckdb/__init__.py:532: error: Argument 1 to "read_parquet" of "Backend" has incompatible type "str | Path | Any"; expected "str | Iterable[str]"  [arg-type]
- ibis/backends/duckdb/__init__.py:536: error: Argument 1 to "read_csv" of "Backend" has incompatible type "str | Any | Path | list[Any] | tuple[Any, ...]"; expected "str | list[str] | tuple[str]"  [arg-type]
+ ibis/backends/duckdb/__init__.py:536: error: Argument 1 to "read_csv" of "Backend" has incompatible type "str | Path | Any"; expected "str | list[str] | tuple[str]"  [arg-type]
- ibis/backends/duckdb/__init__.py:538: error: Argument 1 to "read_postgres" of "Backend" has incompatible type "str | Any | Path | list[Any] | tuple[Any, ...]"; expected "str"  [arg-type]
+ ibis/backends/duckdb/__init__.py:538: error: Argument 1 to "read_postgres" of "Backend" has incompatible type "str | Path | Any"; expected "str"  [arg-type]
- ibis/backends/pyspark/__init__.py:962: error: Argument 1 to "read_parquet" of "Backend" has incompatible type "str | Any | Path | list[Any] | tuple[Any, ...]"; expected "str | Path"  [arg-type]
- ibis/backends/pyspark/__init__.py:966: error: Argument 1 to "read_csv" of "Backend" has incompatible type "str | Any | Path | list[Any] | tuple[Any, ...]"; expected "str | list[str] | tuple[str]"  [arg-type]
+ ibis/backends/pyspark/__init__.py:966: error: Argument 1 to "read_csv" of "Backend" has incompatible type "str | Path | Any"; expected "str | list[str] | tuple[str]"  [arg-type]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

False positive "Statement is unreachable" when checking if typevar is None
1 participant
@sterliakov