gh-96268: Fix loading invalid UTF-8 by mdboom · Pull Request #96270 · python/cpython

mdboom

added 2 commits

August 25, 2022 10:13

This makes tokenizer.c:valid_utf8 match stringlib/codecs.h:decode_utf8.

This also fixes the related test so it will always detect the expected failure
and error message.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request

Sep 7, 2022

This makes tokenizer.c:valid_utf8 match stringlib/codecs.h:decode_utf8.

It also fixes an off-by-one error introduced in 3.10 for the line number when the tokenizer reports bad UTF8.
(cherry picked from commit 8bc356a)

Co-authored-by: Michael Droettboom <mdboom@gmail.com>

miss-islington added a commit that referenced this pull request

Sep 7, 2022

This makes tokenizer.c:valid_utf8 match stringlib/codecs.h:decode_utf8.

It also fixes an off-by-one error introduced in 3.10 for the line number when the tokenizer reports bad UTF8.
(cherry picked from commit 8bc356a)

Co-authored-by: Michael Droettboom <mdboom@gmail.com>