Currently f-strings are parsed just as regular strings by lib2to3. However, this has the problem that the invariant of a string node being a literal is now broken.
This poses two problems. On one hand, if I want to compare that two string nodes are equivalent I would need to do something like:
def string_nodes_are_eqivalent(node1, node2):
if is_f_string(node1) and is_f_string(node2):
# This would require to parse the nodes using ast.parse
return f_strings_are_equivalent(node1, node2)
if not is_f_string(node1) and not is_f_string(node2):
return ast.literal_eval(node1.value) == ast.literal_eval(node2.value)
return False
Note that ast.literal_eval does not accept f-strings.
Also note that ast.parse returns an ast.JoinedString for f-strings, whose elements are either ast.Str or ast.FormattedValue, the latter being ast expressions.
On the other hand, this has the problem of not being able to refactor the expressions inside an f-string.
Note also, that lib2to3 will parse invalid f-strings like f"hello {", whereas ast.parse will raise a SyntaxError exception.
See below for reproduction cases:
In [2]: lib2to3.tests.support.parse_string('f"hello {"')
Out[2]: Node(file_input, [Node(simple_stmt, [Leaf(3, 'f"hello {"'), Leaf(4, '\n')]), Leaf(0, '')])
In [4]: ast.parse('f"hello {"')
Traceback (most recent call last):
File "/Users/skreft/.virtualenvs/pyfaster/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2963, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-4-78a4f2773f7f>", line 1, in <module>
ast.parse('f"hello {"')
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
SyntaxError: f-string: expecting '}'