Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use new type inference as a primary mechanism #15907

Open
ilevkivskyi opened this issue Aug 18, 2023 · 0 comments
Open

Use new type inference as a primary mechanism #15907

ilevkivskyi opened this issue Aug 18, 2023 · 0 comments
Labels
feature priority-1-normal topic-type-variables

Comments

@ilevkivskyi
Copy link
Member

Depends on #15906. This is a follow-up for series started by #15287

Currently new (polymorphic) inference is used as a fallback if the traditional algorithm fails (one way or another). At some point we should switch to the new inference scheme entirely. This would include:

  • Use polymorphic inference during unification in subtyping.
  • Use polymorphic inference as primary mechanism for all inference passes (return type, arguments first pass, arguments second pass).
  • Move apply_poly() logic from checkexpr.py to applytype.py.
@ilevkivskyi ilevkivskyi mentioned this issue Aug 18, 2023
Merged
This was referenced May 26, 2024
Merged
Merged
ilevkivskyi added a commit that referenced this issue Jun 5, 2024
@ilevkivskyi @pre-commit-ci
Use namespaces for function type variables (#17311)
6682563 
Fixes #16582

IMO this is long overdue. Currently, type variable IDs are 99% unique,
but when they accidentally clash, it causes hard to debug issues. The
implementation is generally straightforward, but it uncovered a whole
bunch of unrelated bugs. Few notes:
* This still doesn't fix the type variables in nested generic callable
types (those that appear in return types of another generic callable).
It is non-trivial to put namespace there, and luckily this situation is
already special-cased in `checkexpr.py` to avoid ID clashes.
* This uncovered a bug in overloaded dunder overrides handling, fix is
simple.
* This also uncovered a deeper problem in unsafe overload overlap logic
(w.r.t. partial parameters overlap). Here proper fix would be hard, so
instead I tweak current logic so it will not cause false positives, at a
cost of possible false negatives.
* This makes explicit that we use a somewhat ad-hoc logic for join/meet
of generic callables. FWIW I decided to keep it, since it seems to work
reasonably well.
* This accidentally highlighted two bugs in error message locations. One
very old one related to type aliases, I fixed newly discovered cases by
extending a previous partial fix. Second, the error locations generated
by `partial` plugin were completely off (you can see examples in
`mypy_primer` where there were errors on empty lines etc).
* This PR (naturally) causes a significant amount of new valid errors
(fixed false negatives). To improve the error messages, I extend the
name disambiguation logic to include type variables (and also type
aliases, while I am at it), previously it only applied to `Instance`s.
Note that I use a notation `TypeVar@namespace`, which is a semantic
equivalent of qualified name for type variables. For now, I shorten the
namespace to only the last component, to make errors less verbose. We
can reconsider this if it causes confusion.
* Finally, this PR will hopefully allow a more principled implementation
of #15907

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@ilevkivskyi ilevkivskyi mentioned this issue Jun 8, 2024
Merged
ilevkivskyi added a commit that referenced this issue Jun 8, 2024
@ilevkivskyi
Move apply_type() to applytype.py (#17346)
3518f24 
Moving towards #15907

This is a pure refactoring. It was surprisingly easy, this didn't add
new import cycles, because there is already (somewhat fundamental) cycle
`applytype.py` <-> `subtypes.py`.
@ilevkivskyi ilevkivskyi mentioned this issue Jun 8, 2024
Merged
ilevkivskyi added a commit that referenced this issue Jun 10, 2024
@ilevkivskyi @pre-commit-ci
Use polymorphic inference in unification (#17348)
83d54ff 
Moving towards #15907
Fixes #17206

This PR enables polymorphic inference during unification. This will
allow us to handle even more tricky situations involving generic
higher-order functions (see a random example I added in tests).
Implementation is mostly straightforward, few notes:
* This uncovered another issue with unions in solver, unfortunately
current constraint inference algorithm can sometimes infer weird
constraints like `T <: Union[T, int]`, that later confuse the solver.
* This uncovered another possible type variable clash scenario that was
not handled properly. In overloaded generic function, each overload
should have a different namespace for type variables (currently they all
just get function name). I use `module.some_func#0` etc. for overloads
namespaces instead.
* Another thing with overloads is that the switch caused unsafe overlap
check to change: after some back and forth I am keeping it mostly the
same to avoid possible regressions (unfortunately this requires some
extra refreshing of type variables).
* This makes another `ParamSpec` crash to happen more often so I fix it
in this same PR.
* Finally this uncovered a bug in handling of overloaded `__init__()`
that I am fixing here as well.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature priority-1-normal topic-type-variables
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ilevkivskyi