Regarding the null terminator, I was mainly smoke testing your code. :) Maybe it would be too hard to support properly. Although I could imagine someone doing things like this:
>>> d = {b"key\x00": "value"}
>>> f"key={d[b'key\x00']}" # Oops, escape code at wrong level
File "<fstring>", line 1
(d[b'key
^
SyntaxError: EOL while scanning string literal
>>> rf"key={d[b'key\x00']}" # Corrected
'key=value'
I also finished quickly reading over the C code, with a couple more review comments. But I am not familiar with the files involved to thoroughly review.