Provide hint when Union type with None may not have been narrowed

[angelawuuu]

Fixes #17036

When a union includes None, new users may not know they need an is not None check before accessing attributes. For example, the following code:

class A:
    def foo(self) -> int: pass

def f(x: Union[A, None]) -> int:
    return x.foo()

previously (on the master branch) generated this error message:

main:7: error: Item "None" of "Optional[A]" has no attribute "foo"

which may be ambiguous since x is never explicitly defined as None.

With this PR, the message remains, but we add a suggestion for how to fix the error:

main:7: error: Item "None" of "Optional[A]" has no attribute "foo"
main:7: note: You can use "if x is not None" to guard against a None value

This is possible by checking if None is part of the union when an error signaling a type doesn't have an attribute is triggered and displays the corresponding hint if it is.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

pip (https://github.com/pypa/pip)
+ src/pip/_internal/network/utils.py:63: note: You can use "if response.raw is not None" to guard against a None value
+ src/pip/_internal/network/utils.py:93: note: You can use "if response.raw is not None" to guard against a None value
+ src/pip/_internal/network/auth.py:521: note: You can use "if resp.raw is not None" to guard against a None value
+ src/pip/_internal/network/auth.py:544: note: You can use "if resp.request is not None" to guard against a None value
+ src/pip/_internal/network/session.py:408: note: You can use "if self.auth is not None" to guard against a None value
+ src/pip/_internal/index/collector.py:88: note: You can use "if response.request is not None" to guard against a None value
+ src/pip/_internal/cli/req_command.py:150: note: You can use "if session.auth is not None" to guard against a None value
+ src/pip/_internal/cli/req_command.py:151: note: You can use "if session.auth is not None" to guard against a None value

dragonchain (https://github.com/dragonchain/dragonchain)
+ dragonchain/broadcast_processor/broadcast_functions.py:95:32: note: You can use "if <variable name> is not None" to guard against a None value
+ dragonchain/lib/authorization.py:315:19: note: You can use "if <variable name> is not None" to guard against a None value
+ dragonchain/lib/authorization.py:317:25: note: You can use "if <variable name> is not None" to guard against a None value
+ dragonchain/lib/authorization.py:340:31: note: You can use "if <variable name> is not None" to guard against a None value

check-jsonschema (https://github.com/python-jsonschema/check-jsonschema)
+ src/check_jsonschema/cli/param_types.py:90: note: You can use "if m is not None" to guard against a None value
+ src/check_jsonschema/cli/param_types.py:91: note: You can use "if m is not None" to guard against a None value

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/id.py:558: note: You can use "if session is not None" to guard against a None value

Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/dependencies/limiters.py:819: note: You can use "if resource is not None" to guard against a None value

sphinx (https://github.com/sphinx-doc/sphinx)
+ sphinx/builders/html/__init__.py:441:16: note: You can use "if self._publisher.writer is not None" to guard against a None value

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/_code/code.py:1077: note: You can use "if e is not None" to guard against a None value
+ src/_pytest/_code/code.py:1083: note: You can use "if e is not None" to guard against a None value
+ testing/_py/test_local.py:858: note: You can use "if x is not None" to guard against a None value
+ testing/_py/test_local.py:861: note: You can use "if x is not None" to guard against a None value
+ testing/_py/test_local.py:912: note: You can use "if x is not None" to guard against a None value
+ testing/_py/test_local.py:914: note: You can use "if y is not None" to guard against a None value
+ testing/_py/test_local.py:923: note: You can use "if x is not None" to guard against a None value
+ testing/_py/test_local.py:924: note: You can use "if x is not None" to guard against a None value
+ testing/_py/test_local.py:929: note: You can use "if x is not None" to guard against a None value
+ testing/_py/test_local.py:940: note: You can use "if x is not None" to guard against a None value
+ testing/_py/test_local.py:1281: note: You can use "if cmd is not None" to guard against a None value
+ testing/_py/test_local.py:1283: note: You can use "if cmd is not None" to guard against a None value
+ testing/_py/test_local.py:1284: note: You can use "if x is not None" to guard against a None value

operator (https://github.com/canonical/operator)
+ ops/lib/__init__.py:266: note: You can use "if self.spec.loader is not None" to guard against a None value

kornia (https://github.com/kornia/kornia)
+ kornia/enhance/histogram.py:217: note: You can use "if image is not None" to guard against a None value
+ kornia/enhance/histogram.py:220: note: You can use "if image is not None" to guard against a None value
+ kornia/enhance/histogram.py:241: note: You can use "if image is not None" to guard against a None value
+ kornia/enhance/histogram.py:244: note: You can use "if image is not None" to guard against a None value
+ kornia/enhance/histogram.py:249: note: You can use "if image is not None" to guard against a None value
+ kornia/enhance/histogram.py:251: note: You can use "if image is not None" to guard against a None value

werkzeug (https://github.com/pallets/werkzeug)
+ tests/test_wrappers.py:179: note: You can use "if a is not None" to guard against a None value
+ tests/test_wrappers.py:180: note: You can use "if a is not None" to guard against a None value
+ tests/test_wrappers.py:181: note: You can use "if a is not None" to guard against a None value
+ tests/test_wrappers.py:189: note: You can use "if a is not None" to guard against a None value
+ tests/test_wrappers.py:190: note: You can use "if a is not None" to guard against a None value
+ tests/test_wrappers.py:191: note: You can use "if a is not None" to guard against a None value
+ tests/test_wrappers.py:1078: note: You can use "if req.range is not None" to guard against a None value
+ tests/test_wrappers.py:1081: note: You can use "if req.range is not None" to guard against a None value
+ tests/test_test.py:166: note: You can use "if b.input_stream is not None" to guard against a None value
+ tests/test_test.py:168: note: You can use "if b.input_stream is not None" to guard against a None value
+ tests/test_test.py:306: note: You can use "if request.authorization is not None" to guard against a None value
+ tests/test_test.py:307: note: You can use "if request.authorization is not None" to guard against a None value
+ tests/test_routing.py:601: note: You can use "if rule.methods is not None" to guard against a None value
+ tests/test_routing.py:603: note: You can use "if rule.methods is not None" to guard against a None value
+ tests/test_routing.py:1310: note: You can use "if None is not None" to guard against a None value
+ tests/test_local.py:191: note: You can use "if local.LocalProxy.__doc__ is not None" to guard against a None value
+ tests/test_http.py:152: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:153: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:154: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:157: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:158: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:159: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:162: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:163: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:164: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:176: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:177: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:178: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:179: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:180: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:181: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:182: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:183: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:184: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:185: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:195: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:196: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:197: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:198: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:199: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:200: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:201: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:205: note: You can use "if <variable name> is not None" to guard against a None value
+ tests/test_http.py:211: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:212: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:217: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:218: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:242: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:243: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:244: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:245: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:253: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:254: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:255: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:256: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:257: note: You can use "if wa is not None" to guard against a None value
+ tests/test_http.py:259: note: You can use "if <variable name> is not None" to guard against a None value
+ tests/test_http.py:266: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:267: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:272: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:273: note: You can use "if a is not None" to guard against a None value
+ tests/test_http.py:623: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:624: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:625: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:628: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:629: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:630: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:633: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:634: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:635: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:638: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:639: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:640: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:643: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:644: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:645: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:664: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:665: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:666: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:667: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:668: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:677: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:678: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:679: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:680: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:681: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:684: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:685: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:686: note: You can use "if rv is not None" to guard against a None value
+ tests/test_http.py:687: note: You can use "if rv is not None" to guard against a None value
+ tests/test_debug.py:281: note: You can use "if e.__context__ is not None" to guard against a None value
+ tests/test_datastructures.py:1189: note: You can use "if r is not None" to guard against a None value
+ tests/conftest.py:86: note: You can use "if self is not None" to guard against a None value
+ tests/conftest.py:123: note: You can use "if client.log is not None" to guard against a None value

discord.py (https://github.com/Rapptz/discord.py)
+ discord/guild.py:3232: note: You can use "if channel is not None" to guard against a None value

streamlit (https://github.com/streamlit/streamlit)
+ lib/tests/streamlit/streamlit_test.py:61:30: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/scriptrunner/script_requests_test.py:116:32: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/scriptrunner/script_requests_test.py:119:31: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/scriptrunner/script_requests_test.py:141:32: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/scriptrunner/script_requests_test.py:142:31: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/scriptrunner/script_requests_test.py:163:32: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/scriptrunner/script_requests_test.py:164:31: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/logger_test.py:104:21: note: You can use "if None is not None" to guard against a None value
+ lib/tests/streamlit/logger_test.py:108:21: note: You can use "if None is not None" to guard against a None value
+ lib/tests/streamlit/config_test.py:402:23: note: You can use "if config._config_options is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:136:27: note: You can use "if original_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:139:9: note: You can use "if original_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:145:9: note: You can use "if original_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:146:9: note: You can use "if original_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:153:9: note: You can use "if original_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:154:9: note: You can use "if original_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:163:16: note: You can use "if reconnected_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:163:52: note: You can use "if original_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:165:16: note: You can use "if reconnected_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:168:16: note: You can use "if reconnected_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:178:16: note: You can use "if active_session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:219:9: note: You can use "if session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:234:9: note: You can use "if session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:243:16: note: You can use "if session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/websocket_session_manager_test.py:253:16: note: You can use "if session_info is not None" to guard against a None value
+ lib/tests/streamlit/runtime/runtime_test.py:233:19: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/runtime_test.py:253:19: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/runtime_test.py:342:27: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/runtime_test.py:370:23: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/runtime_test.py:396:23: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/app_session_test.py:488:9: note: You can use "if session._local_sources_watcher is not None" to guard against a None value
+ lib/tests/streamlit/runtime/scriptrunner/script_runner_test.py:1138:24: note: You can use "if widget is not None" to guard against a None value
+ lib/tests/streamlit/form_test.py:251:33: note: You can use "if form_data is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:310:26: note: You can use "if dg._cursor is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:311:26: note: You can use "if dg._cursor is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:317:25: note: You can use "if dg._cursor is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:318:26: note: You can use "if dg._cursor is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:336:26: note: You can use "if c1 is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:336:46: note: You can use "if c2 is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:337:26: note: You can use "if c1 is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:337:37: note: You can use "if c2 is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:338:26: note: You can use "if c1 is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:338:43: note: You can use "if c2 is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:339:26: note: You can use "if c1 is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:339:37: note: You can use "if c2 is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:350:29: note: You can use "if dg._cursor is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:359:29: note: You can use "if dg._cursor is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:370:31: note: You can use "if dg._cursor is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:388:9: note: You can use "if ctx is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:398:9: note: You can use "if ctx is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:399:9: note: You can use "if ctx is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:407:9: note: You can use "if ctx is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:408:9: note: You can use "if ctx is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:423:26: note: You can use "if container._cursor is not None" to guard against a None value
+ lib/tests/streamlit/delta_generator_test.py:442:30: note: You can use "if column._cursor is not None" to guard against a None value
+ lib/tests/streamlit/web/server/server_test.py:215:26: note: You can use "if ws_client.headers is not None" to guard against a None value
+ lib/tests/streamlit/web/server/server_test.py:231:31: note: You can use "if ws_client.headers is not None" to guard against a None value
+ lib/tests/streamlit/runtime/metrics_util_test.py:179:9: note: You can use "if ctx is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/widgets_test.py:473:17: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/widgets_test.py:480:17: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/widgets_test.py:488:17: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/widgets_test.py:495:17: note: You can use "if <variable name> is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:177:16: note: You can use "if serialized is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:178:16: note: You can use "if serialized is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:182:16: note: You can use "if serialized is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:183:16: note: You can use "if serialized is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:200:16: note: You can use "if serialized is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:201:21: note: You can use "if serialized is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:215:16: note: You can use "if serialized is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:216:16: note: You can use "if serialized is not None" to guard against a None value
+ lib/tests/streamlit/runtime/state/session_state_test.py:239:9: note: You can use "if metadata.callback is not None" to guard against a None value

... (truncated 1 lines) ...

ibis (https://github.com/ibis-project/ibis)
+ ibis/expr/types/generic.py:1225: note: You can use "if data_mapper is not None" to guard against a None value
+ ibis/expr/types/generic.py:1395: note: You can use "if data_mapper is not None" to guard against a None value
+ ibis/expr/types/relations.py:232: note: You can use "if data_mapper is not None" to guard against a None value
+ ibis/expr/types/relations.py:1149: note: You can use "if metrics is not None" to guard against a None value
+ ibis/expr/types/relations.py:1150: note: You can use "if metrics is not None" to guard against a None value
+ ibis/expr/types/relations.py:3833: note: You can use "if match_result is not None" to guard against a None value
+ ibis/expr/types/relations.py:4248: note: You can use "if names is not None" to guard against a None value
+ ibis/expr/types/relations.py:4254: note: You can use "if names is not None" to guard against a None value
+ ibis/expr/types/relations.py:4256: note: You can use "if names is not None" to guard against a None value
+ ibis/backends/sql/datatypes.py:1030: note: You can use "if scale is not None" to guard against a None value
+ ibis/backends/tests/base.py:104: note: You can use "if <variable name> is not None" to guard against a None value
+ ibis/examples/gen_registry.py:328: note: You can use "if <variable name> is not None" to guard against a None value
+ ibis/backends/druid/tests/conftest.py:87: note: You can use "if match is not None" to guard against a None value
+ ibis/backends/datafusion/__init__.py:290: note: You can use "if catalog is not None" to guard against a None value
+ ibis/backends/datafusion/__init__.py:292: note: You can use "if catalog is not None" to guard against a None value
+ ibis/backends/datafusion/__init__.py:294: note: You can use "if database is not None" to guard against a None value
+ ibis/backends/clickhouse/__init__.py:258: note: You can use "if expr is not None" to guard against a None value

[kaste]

IMO, I would not add a note at all in the "if <variable name> is not None"-case. E.g. code like re.search("^DC(.*)-HMAC", authorization).group(1) would produce such a note.

[brianschubert]

This branch has some hiccups if the expression isn't strictly a dotted member expression:

class A:
    x: int | None

class B:
    a: A

[A()][0].x.bit_count()  # N: You can use "if None is not None" to guard against a None value
B().a.x.bit_count()     # N: You can use "if None.x is not None" to guard against a None value

I think two adjustments would be needed:

1. check for get_member_expr_fullname returning None (and don't emit the note if it does)
2. fix the case where get_member_expr_fullname returns a string with an embedded "None" (edit: fixed in #17848)