Add support for spec #14739

hauntsaninja · 2023-02-20T01:02:21Z

It was too annoying to get the fixtures to work out here, so I didn't. But you can see the tests will add the None for the non __main__ case

Co-authored-by: Joongi Kim [email protected]

Fixes python#4145 It was too annoying to get the fixtures to work out here, so I didn't. But you can see the tests will add the None for the non `__main__` case

JukkaL · 2023-02-21T16:19:02Z

mypy/semanal.py

+            elif name == "__spec__":
+                inst: Type | None = self.named_type_or_none("importlib.machinery.ModuleSpec")
+                if inst is None:
+                    if self.final_iteration:


I wonder if this may down slow down tests significantly, since every module top level will have to be processed many times? Maybe instead check options.use_builtins_fixtures and fall back to builtins.object immediately if it's true.

JukkaL · 2023-02-21T16:19:56Z

test-data/unit/check-basic.test

+reveal_type(module.__name__)  # N: Revealed type is "builtins.str"
+# This will actually reveal importlib.machinery.ModuleSpec
+reveal_type(module.__spec__)  # N: Revealed type is "builtins.object"
+[file module.py]


Consider also adding a pythoneval test that uses full stubs.

achimnol · 2024-05-18T20:11:06Z

@hauntsaninja @JukkaL Is there anything that I could help to conclude this PR?
It seems not to be too difficult but I'm just not familiar enough with the mypy development.
Maybe somebody could help me in the PyCon US sprint?

hauntsaninja · 2024-05-18T20:26:02Z

Yeah, that sounds good. And if you want to experiment before sprints, feel free to post a diff on this PR and I can commit and merge it :-)

achimnol · 2024-05-19T05:59:21Z

I'd like to, but I need some guidance on writing tests in mypy and internal mechanisms mentioned in the previous review. I'll try a look.

achimnol · 2024-05-20T05:19:31Z

Maybe the pythoneval test update could be:

diff --git a/test-data/unit/pythoneval.test b/test-data/unit/pythoneval.test
index 0ed3540b6..9a43d9071 100644
--- a/test-data/unit/pythoneval.test
+++ b/test-data/unit/pythoneval.test
@@ -107,14 +107,20 @@ f
 [case testModuleAttributes]
 import math
 import typing
+print(type(__spec__))
 print(math.__name__)
+print(math.__spec__.name)
 print(type(math.__dict__))
 print(type(math.__doc__ or ''))
+print(type(math.__spec__))
 print(math.__class__)
 [out]
+<class 'NoneType'>
+math
 math
 <class 'dict'>
 <class 'str'>
+<class '_frozen_importlib.ModuleSpec'>
 <class 'module'>

 [case testSpecialAttributes]

achimnol · 2024-05-20T05:34:43Z

Maybe the first review would intend the following changes..?

diff --git a/mypy/semanal.py b/mypy/semanal.py
index 5bafe124b..1158d7f7b 100644
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -662,14 +662,14 @@ class SemanticAnalyzer(
                     return
                 typ = inst
             elif name == "__spec__":
-                inst: Type | None = self.named_type_or_none("importlib.machinery.ModuleSpec")
+                if self.options.use_builtins_fixtures:
+                    inst = self.named_type_or_none("builtins.object")
+                else:
+                    inst = self.named_type_or_none("importlib.machinery.ModuleSpec")
                 if inst is None:
-                    if self.final_iteration:
-                        inst = self.named_type_or_none("builtins.object")
-                        assert inst is not None, "Cannot find builtins.object"
-                    else:
-                        self.defer()
-                        return
+                    assert not self.final_iteration, "Cannot find builtins.object to add __spec__"
+                    self.defer()
+                    return
                 if file_node.name == "__main__":
                     # https://docs.python.org/3/reference/import.html#main-spec
                     inst = UnionType.make_union([inst, NoneType()])

hauntsaninja · 2024-05-20T14:24:12Z

Thank you, pushed with some minor changes (I think we still need the final_iteration code)! Btw, I'm in Room 320 :-)

achimnol · 2024-05-20T14:54:03Z

Thanks!!! 👍🏼

achimnol · 2024-05-20T15:54:00Z

@hauntsaninja Here is another patch to add:

diff --git a/test-data/unit/fine-grained-inspect.test b/test-data/unit/fine-grained-inspect.test
index f8ce35585..ed89f2f09 100644
--- a/test-data/unit/fine-grained-inspect.test
+++ b/test-data/unit/fine-grained-inspect.test
@@ -236,7 +236,7 @@ class C: ...
 [builtins fixtures/module.pyi]
 [out]
 ==
-{"<pack.bar>": ["C", "__annotations__", "__doc__", "__file__", "__name__", "__package__", "bar", "x"], "ModuleType": ["__file__", "__getattr__"]}
+{"<pack.bar>": ["C", "__annotations__", "__doc__", "__file__", "__name__", "__package__", "__spec__", "bar", "x"], "ModuleType": ["__file__", "__getattr__"]}

 [case testInspectModuleDef]
 # inspect2: --show=definition --include-kind tmp/foo.py:2:1

achimnol · 2024-05-20T15:54:21Z

~~I'm not sure what's happening with pydantic though.... 🤔~~

It seems to be the expected result as now mypy recognizes the __spec__ variable.

achimnol · 2024-05-20T21:49:38Z

test-data/unit/pythoneval.test

 math
+math
+<class '_frozen_importlib_external.ExtensionFileLoader'>


This breaks in Python 3.8... 😥

github-actions · 2024-05-20T22:37:54Z

Diff from mypy_primer, showing the effect of this PR on open source code:

pydantic (https://github.com/samuelcolvin/pydantic)
- pydantic/__init__.py:223: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:225: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:226: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:227: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:228: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:229: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:230: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:231: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:232: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:234: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:236: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:237: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:238: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:239: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:240: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:242: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:243: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:245: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:247: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:248: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:249: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:250: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:251: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:252: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:254: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:255: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:256: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:258: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:259: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:260: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:262: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:263: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:265: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:266: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:267: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:268: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:269: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:270: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:271: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:272: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:273: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:274: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:275: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:276: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:277: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:278: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:279: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:280: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:281: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:282: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:283: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:284: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:285: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:286: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:287: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:288: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:290: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:292: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:293: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:294: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:295: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:296: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:297: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:298: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:299: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:300: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:301: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:302: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:303: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:304: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:305: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:306: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:307: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:308: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:309: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:310: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:311: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:312: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:313: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:314: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:315: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:316: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:317: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:318: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:319: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:320: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:321: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:322: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:323: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:324: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:325: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:326: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:327: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:328: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:329: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:330: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:331: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:332: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:333: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:334: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:335: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:336: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:337: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:338: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:339: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:340: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:341: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:342: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:343: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:344: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:345: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:346: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:347: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:348: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:349: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:350: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:352: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:354: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:355: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:356: error: Name "__spec__" is not defined  [name-defined]
- pydantic/__init__.py:358: error: Name "__spec__" is not defined  [name-defined]

... (truncated 139 lines) ...

Add support for __spec__

830db42

Fixes python#4145 It was too annoying to get the fixtures to work out here, so I didn't. But you can see the tests will add the None for the non `__main__` case

This comment has been minimized.

Sign in to view

hauntsaninja marked this pull request as draft February 20, 2023 01:57

JukkaL reviewed Feb 21, 2023

View reviewed changes

feedback

50d0897

hauntsaninja and others added 2 commits May 20, 2024 07:36

Merge branch 'master' into spec

83fefbf

fixup

4705f0b

hauntsaninja marked this pull request as ready for review May 20, 2024 14:38

This comment has been minimized.

Sign in to view

.

411c328

This comment has been minimized.

Sign in to view

achimnol reviewed May 20, 2024

View reviewed changes

.

d01b0cf

hauntsaninja merged commit f5afdcd into python:master May 21, 2024
18 checks passed

hauntsaninja deleted the spec branch May 21, 2024 03:35

privatedumbo mentioned this pull request Jun 1, 2024

fix: Feast UI importlib change feast-dev/feast#4248

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for spec #14739

Add support for spec #14739

hauntsaninja commented Feb 20, 2023 •

edited

Loading

This comment has been minimized.

JukkaL Feb 21, 2023

JukkaL Feb 21, 2023

achimnol commented May 18, 2024 •

edited

Loading

hauntsaninja commented May 18, 2024

achimnol commented May 19, 2024

achimnol commented May 20, 2024 •

edited

Loading

achimnol commented May 20, 2024

hauntsaninja commented May 20, 2024

achimnol commented May 20, 2024

This comment has been minimized.

achimnol commented May 20, 2024

achimnol commented May 20, 2024 •

edited

Loading

This comment has been minimized.

achimnol May 20, 2024

github-actions bot commented May 20, 2024

Add support for __spec__ #14739

Add support for __spec__ #14739

Conversation

hauntsaninja commented Feb 20, 2023 • edited Loading

This comment has been minimized.

JukkaL Feb 21, 2023

Choose a reason for hiding this comment

JukkaL Feb 21, 2023

Choose a reason for hiding this comment

achimnol commented May 18, 2024 • edited Loading

hauntsaninja commented May 18, 2024

achimnol commented May 19, 2024

achimnol commented May 20, 2024 • edited Loading

achimnol commented May 20, 2024

hauntsaninja commented May 20, 2024

achimnol commented May 20, 2024

This comment has been minimized.

achimnol commented May 20, 2024

achimnol commented May 20, 2024 • edited Loading

This comment has been minimized.

achimnol May 20, 2024

Choose a reason for hiding this comment

github-actions bot commented May 20, 2024

Add support for spec #14739

Add support for spec #14739

hauntsaninja commented Feb 20, 2023 •

edited

Loading

achimnol commented May 18, 2024 •

edited

Loading

achimnol commented May 20, 2024 •

edited

Loading

achimnol commented May 20, 2024 •

edited

Loading