bpo-23894: Make rb'' strings work in lib2to3 by ambv · Pull Request #1724 · python/cpython
This partially solves bpo-23894. A separate PR coming later to support f-strings.
Make rb'' strings work in lib2to3
bpo-23894: Make rb'' strings work in lib2to3
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"String" didn't previously include b but this is suspicious to me. Why wouldn't it? That way b'' literals were only PseudoTokens (via ContStr), not actual Tokens (via String).
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LG
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding the test: LGTM. As far as the tokenizer changes, I'm not familiar enough with lib2to3 to provide a useful review. Assuming other reviewers have no objections, I'm fine for this going into 3.6.2rc.
Tests are failing because I used an f-string in the test and lib2to3 is parsing that test file as part of fixers. f-strings are not supported yet in lib2to3. Switching to using a classic str.format fixes it. I'm updating the PR now.
UPDATE: as expected, tests pass now.
This partially solves bpo-23894.
ambv added a commit to ambv/cpython that referenced this pull request
This partially solves bpo-23894. (cherry picked from commit 0c4aca5)
I was requested for a review, but after I completed my review, I see that the change was already merged :-D It's fine, don't worry. I just want to give my post-commit LGTM! Nice change. Thanks @ambv for taking care of 2to3 ;-)