◐ Shell
clean mode source ↗

Message 366165 - Python tracker

At the moment, if the prefix is invalid, it gets tokenized as a NAME node and the string following is tokenized as a STRING node with no prefix.

╰─ echo "ur''" | ./python -m tokenize
1,0-1,2:            NAME           'ur'           
1,2-1,4:            STRING         "''"           
1,4-1,5:            NEWLINE        '\n'           
2,0-2,0:            ENDMARKER      ''

╰─ echo "f''" | ./python -m tokenize  
1,0-1,3:            STRING         "f''"          
1,3-1,4:            NEWLINE        '\n'           
2,0-2,0:            ENDMARKER      ''