bpo-43224: Implement PEP 646 grammar changes by mrahtz · Pull Request #31018 · python/cpython

the-knights-who-say-ni

I just fixed a merge conflict and a whitespace issue found by make patchcheck. I think we can go ahead and merge, but I noticed a few things we still need to address:

(1) The ast.py changes aren't tested. They should be tested with a call to ast.unparse.

(2) There is also a C version of unparse in Python/ast_unparse.c, which is used for from __future__ import annotations. It likely needs changes too.

(3) If you use from __future__ import annotations and call get_type_hints() on a function that takes *args: *Ts, it fails because it tries to parse *Ts as an expression. I guess to solve this we'll have to detect an initial * and remove it before trying to parse the code.

Here's an example that demonstrates these issues:

Details

from __future__ import annotations
import typing
import ast

Ts = typing.TypeVarTuple("Ts")

def f(x: tuple[*Ts], *args: *Ts):
    pass

print(f.__annotations__)
print(ast.unparse(ast.parse("tuple[*Ts]")))
print(typing.get_type_hints(f))

% ./python.exe Doc/tools/annotvt.py
{'x': 'tuple[(*Ts,)]', 'args': '*Ts'}
tuple[*Ts,]
Traceback (most recent call last):
  File "/Users/jelle/py/cpython/Lib/typing.py", line 738, in __init__
    code = compile(arg, '<string>', 'eval')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1
    *Ts
    ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jelle/py/cpython/Doc/tools/annotvt.py", line 12, in <module>
    print(typing.get_type_hints(f))
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jelle/py/cpython/Lib/typing.py", line 2199, in get_type_hints
    value = ForwardRef(
            ^^^^^^^^^^^
  File "/Users/jelle/py/cpython/Lib/typing.py", line 740, in __init__
    raise SyntaxError(f"Forward reference must be an expression -- got {arg!r}")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: Forward reference must be an expression -- got '*Ts'