+1 in general to this work. Łukasz is effectively preaching to the choir by looping me in here. :)
It is a big challenge to practically support Python in that we have no good ability to parse and understand all language syntax versions via a single API that does not depend on the version of the language your own tools process is running under.
lib2to3.pgen2 is the closest thing we've got and it used by a notable crop of python refactoring tools today because there really wasn't another available choice. All they know is that they've got a ".py" file, they can't know which specific language versions it may be intended for. Nor should they ever need to run _on_ that language version. That situation is a nightmare (ex: pylint uses ast and must run on the version of the language it is to analyze as)
I'd love to see a ponycorn module that everything could use to run on top of Python 3.recent yet be able to meaningfully process 2.7 and 3.4-3.7 code. This is an area where the language versions we support parsing and analyzing should _not_ be limited to the current CPython org still supported releases.
Does this need to go in the CPython project and integrate with its internals such as pgen.c or pgen2? I don't know. From my perspective this could be a PyPI project. Even if it seems odd that we have stdlib ast and lib2to3.pgen2 modules and pgen internal to CPython; at some point those could be seen as implementation details and made private in favor of tool application code using a canonical ponycorn thing on PyPI. The important part is that it is maintained and kept up to date with future language grammar changes while maintaining "backwards grammar compatibility".