bpo-41249: Fix postponed annotations for TypedDict#27017

Kronuz

Summary:

This fixes TypedDict to work with get_type_hints() and postponed evaluation of annotations across modules.

This is done by adding the module name to ForwardRef at the time the object is created and using that to resolve the globals during the evaluation.

Test Plan:

Having these two file:

# foo.py
from __future__ import annotations

from typing import Optional, TypedDict

class Foo(TypedDict):
    a: Optional[int]

# bar.py
from __future__ import annotations

from typing import get_type_hints
from foo import Foo

class Bar(Foo, total=False):
    b: int

print(get_type_hints(Bar))

Before:

$ python bar.py
Traceback (most recent call last):
  File "/Users/kronuz/Development/cpython/bar.py", line 9, in <module>
    print(get_type_hints(Bar))
  File "/Users/kronuz/Development/cpython/Lib/typing.py", line 1724, in get_type_hints
    value = _eval_type(value, base_globals, base_locals)
  File "/Users/kronuz/Development/cpython/Lib/typing.py", line 317, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "/Users/kronuz/Development/cpython/Lib/typing.py", line 660, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'Optional' is not defined

After:

$ python bar.py
{'a': typing.Optional[int], 'b': <class 'int'>}

https://bugs.python.org/issue41249

Fidget-Spinner

Thanks for submitting this fix! Some general comments:

Please add your proposed tests. Tests for typing are at https://github.com/python/cpython/blob/main/Lib/test/test_typing.py. Since your tests require multiple files, you can create them in that folder, or reuse this file that's used for testing type annotations in separate modules https://github.com/python/cpython/blob/main/Lib/test/mod_generics_cache.py)
Please add a news entry via blurb.

I'll try to do a full review later.

Kronuz

@Fidget-Spinner, I added the tests, the news entry and fixed the KeyError as suggested. Thank you!

Fidget-Spinner

LG in general! I just have a few non-code comments. Thanks :).

This fixes TypedDict to work with get_type_hints and postponed evaluation of annotations across modules.

Fidget-Spinner

LGTM. Thanks for the quick response!

Please don't force push in the future. It overwrites valuable history, and besides, the core devs will squash and merge your PR as a single commit :).

I can't merge your PR, so I'll wait for Guido or some other core dev to review. If no one tends to this in a week or two, please gently ping the issue tracker.

Kronuz

@Fidget-Spinner, I made the suggested changes. Thank you for reviewing! 😃

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>

gvanrossum

LG except I'd like the helper module renamed. Thanks!

septatrix

I still think it would be beneficial to figure out why TypedDicts have no MRO as this would trivially solve this problem. That being said I think this is a rather clean workaround for the time being

gvanrossum

Just one nit, I’ll commit the fix and then land.

Fidget-Spinner

@Kronuz congrats on landing your first PR in CPython :)! Great work!

gvanrossum

Yes, good work!

miss-islington

Thanks @Kronuz for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington

Thanks @Kronuz for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

miss-islington

Sorry, @Kronuz and @gvanrossum, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 889036f7ef7290ef15b6c3373023f6a35387af0c 3.9

miss-islington

Sorry @Kronuz and @gvanrossum, I had trouble checking out the 3.10 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 889036f7ef7290ef15b6c3373023f6a35387af0c 3.10

Fidget-Spinner

I'll look into it.

miss-islington

Thanks @Kronuz for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

bedevere-bot

GH-27204 is a backport of this pull request to the 3.10 branch.

This fixes TypedDict to work with get_type_hints and postponed evaluation of annotations across modules. This is done by adding the module name to ForwardRef at the time the object is created and using that to resolve the globals during the evaluation. Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> (cherry picked from commit 889036f) Co-authored-by: Germán Méndez Bravo <german.mb@gmail.com>

Fidget-Spinner

Seems like bedevere was just feeling a little unwell ;).

This fixes TypedDict to work with get_type_hints and postponed evaluation of annotations across modules. This is done by adding the module name to ForwardRef at the time the object is created and using that to resolve the globals during the evaluation. Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>

This fixes TypedDict to work with get_type_hints and postponed evaluation of annotations across modules. This is done by adding the module name to ForwardRef at the time the object is created and using that to resolve the globals during the evaluation. Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> (cherry picked from commit 889036f) Co-authored-by: Germán Méndez Bravo <german.mb@gmail.com>

…H-27205) This fixes TypedDict to work with get_type_hints and postponed evaluation of annotations across modules. This is done by adding the module name to ForwardRef at the time the object is created and using that to resolve the globals during the evaluation. Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Co-authored-by: Germán Méndez Bravo <german.mb@gmail.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl>

Kronuz requested review from gvanrossum and ilevkivskyi as code owners July 4, 2021 16:49

the-knights-who-say-ni added the CLA signed label Jul 4, 2021

bedevere-bot added the awaiting review label Jul 4, 2021

Fidget-Spinner reviewed Jul 4, 2021

View reviewed changes

Kronuz force-pushed the main branch from a760c95 to 241247b Compare July 4, 2021 18:42

Kronuz requested a review from Fidget-Spinner July 6, 2021 21:04

Fidget-Spinner reviewed Jul 7, 2021

View reviewed changes

Fix postponed annotations for TypedDict …

32179f2

This fixes TypedDict to work with get_type_hints and postponed evaluation of annotations across modules.

Kronuz force-pushed the main branch from 241247b to 32179f2 Compare July 7, 2021 16:37

Kronuz requested a review from Fidget-Spinner July 7, 2021 16:42

Fidget-Spinner approved these changes Jul 7, 2021

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels Jul 7, 2021

Kronuz and others added 2 commits July 7, 2021 09:52

Remove pull request number from comment …

387f16e

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>

Formatting in the NEWS …

846b7c2

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>

gvanrossum reviewed Jul 7, 2021

View reviewed changes

Change summary and test helper file

fa412f3

septatrix reviewed Jul 8, 2021

View reviewed changes

Fix module import

659fba3

gvanrossum approved these changes Jul 17, 2021

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting core review labels Jul 17, 2021

Update module name in docstring.

1ed104a

gvanrossum merged commit 889036f into python:main Jul 17, 2021

bedevere-bot removed the awaiting merge label Jul 17, 2021

gvanrossum added needs backport to 3.9 needs backport to 3.10 only security fixes labels Jul 17, 2021

miss-islington assigned gvanrossum Jul 17, 2021

Fidget-Spinner added needs backport to 3.10 only security fixes and removed needs backport to 3.10 only security fixes needs backport to 3.9 labels Jul 17, 2021

bedevere-bot removed the needs backport to 3.10 only security fixes label Jul 17, 2021

pablogsal mentioned this pull request Jul 18, 2021

[3.10] Correct the order of regen-abidump #27228

Closed

carljm mentioned this pull request Nov 11, 2022

Fix get_type_hints() on x-module inherited TypedDict in 3.9 and 3.10 python/typing_extensions#94

Merged

Conversation

Kronuz commented Jul 4, 2021 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary:

Test Plan:

Before:

After:

Uh oh!

Fidget-Spinner left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kronuz commented Jul 4, 2021

Uh oh!

Fidget-Spinner left a comment

Choose a reason for hiding this comment

Uh oh!

Fidget-Spinner left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kronuz commented Jul 7, 2021

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

septatrix left a comment

Choose a reason for hiding this comment

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

Fidget-Spinner commented Jul 17, 2021

Uh oh!

gvanrossum commented Jul 17, 2021

Uh oh!

miss-islington commented Jul 17, 2021

Uh oh!

miss-islington commented Jul 17, 2021

Uh oh!

miss-islington commented Jul 17, 2021

Uh oh!

miss-islington commented Jul 17, 2021

Uh oh!

Fidget-Spinner commented Jul 17, 2021

Uh oh!

miss-islington commented Jul 17, 2021

Uh oh!

bedevere-bot commented Jul 17, 2021

Uh oh!

Fidget-Spinner commented Jul 17, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Kronuz commented Jul 4, 2021 •

edited by bedevere-bot

Loading

Fidget-Spinner left a comment •

edited

Loading

Fidget-Spinner left a comment •

edited

Loading