◐ Shell
reader mode source ↗
Skip to content

gh-52551: Fix encoding issues in strftime()#125193

Merged
serhiy-storchaka merged 5 commits into
python:mainfrom
serhiy-storchaka:strftime
Oct 17, 2024
Merged

gh-52551: Fix encoding issues in strftime()#125193
serhiy-storchaka merged 5 commits into
python:mainfrom
serhiy-storchaka:strftime

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Oct 9, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Fix time.strftime(), the strftime() method and formatting of the datetime classes datetime, date and time.

  • Characters not encodable in the current locale are now acceptable in the format string.
  • Surrogate pairs and sequence of surrogatescape-encoded bytes are no longer recombinated.
  • Embedded null character no longer terminates the format string.

Closes #52551, #78662 and #124531.

@serhiy-storchaka
pythongh-52551: Fix encoding issues in strftime() …
c9f1156 
Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.

* Characters not encodable in the current locale are now acceptable in
  the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
  longer recombinated.
* Embedded null character no longer terminates the format string.

This fixes also pythongh-78662 and pythongh-124531.
@serhiy-storchaka serhiy-storchaka added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes labels Oct 9, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Oct 9, 2024
Closed
This was linked to issues Oct 9, 2024
time.strftime() and Unicode characters on Windows #52551
Closed
datetime.strftime strings can be terminated by "\x00" literals #124531
Closed
@serhiy-storchaka serhiy-storchaka changed the title gh-52551: Fix encoding issues in strftime() Oct 9, 2024
@serhiy-storchaka serhiy-storchaka changed the title gh-52551, gh-78662, gh-124531: Fix encoding issues in strftime() Oct 9, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Oct 9, 2024
Closed
@serhiy-storchaka serhiy-storchaka changed the title gh-124531: Fix encoding issues in strftime() Oct 9, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Oct 9, 2024
Closed
@serhiy-storchaka serhiy-storchaka changed the title gh-78662: Fix encoding issues in strftime() Oct 9, 2024
@serhiy-storchaka serhiy-storchaka mentioned this pull request Oct 9, 2024
Closed
@serhiy-storchaka

serhiy-storchaka commented Oct 10, 2024

Copy link
Copy Markdown
Member Author

@vstinner, could you please review this PR?

@serhiy-storchaka

serhiy-storchaka commented Oct 17, 2024

Copy link
Copy Markdown
Member Author

I addressed all comments. Perhaps it is now more difficult to review.

vstinner
vstinner approved these changes Oct 17, 2024
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

LGTM. I just left minor coding style suggestions that you can ignore.

Thanks for the exhaustive added tests, it's very helpful.

@serhiy-storchaka

serhiy-storchaka commented Oct 17, 2024

Copy link
Copy Markdown
Member Author

Thank you for your review @vstinner.

@serhiy-storchaka serhiy-storchaka enabled auto-merge (squash) October 17, 2024 15:19
@serhiy-storchaka serhiy-storchaka merged commit ad3eac1 into python:main Oct 17, 2024
@miss-islington-app

miss-islington-app Bot commented Oct 17, 2024

Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

@miss-islington-app

miss-islington-app Bot commented Oct 17, 2024

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker ad3eac1963a5f195ef9b2c1dbb5e44fa3cce4c72 3.13

@miss-islington-app

miss-islington-app Bot commented Oct 17, 2024

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker ad3eac1963a5f195ef9b2c1dbb5e44fa3cce4c72 3.12

@serhiy-storchaka serhiy-storchaka deleted the strftime branch October 17, 2024 16:18
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Oct 17, 2024
@serhiy-storchaka
[3.13] pythongh-52551: Fix encoding issues in strftime() (pythonGH-12… …
13ef787 
…5193)

Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.

* Characters not encodable in the current locale are now acceptable in
  the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
  longer recombinated.
* Embedded null character no longer terminates the format string.

This fixes also pythongh-78662 and pythongh-124531.
(cherry picked from commit ad3eac1)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Oct 17, 2024
@serhiy-storchaka
pythongh-52551: Fix encoding issues in strftime() (pythonGH-125193) …
4019173 
Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.

* Characters not encodable in the current locale are now acceptable in
  the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
  longer recombinated.
* Embedded null character no longer terminates the format string.

This fixes also pythongh-78662 and pythongh-124531.

(cherry picked from commit ad3eac1)
@bedevere-app

bedevere-app Bot commented Oct 17, 2024

Copy link
Copy Markdown

GH-125657 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Oct 17, 2024
@serhiy-storchaka serhiy-storchaka removed the needs backport to 3.12 only security fixes label Oct 17, 2024
@serhiy-storchaka serhiy-storchaka removed their assignment Oct 17, 2024
serhiy-storchaka added a commit that referenced this pull request Oct 17, 2024
@serhiy-storchaka
[3.13] gh-52551: Fix encoding issues in strftime() (GH-125193) (GH-12… …
08ccbb9 
…5657)

Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.

* Characters not encodable in the current locale are now acceptable in
  the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
  longer recombinated.
* Embedded null character no longer terminates the format string.

This fixes also gh-78662 and gh-124531.

(cherry picked from commit ad3eac1)
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 17, 2024
@serhiy-storchaka @miss-islington
[3.13] pythongh-52551: Fix encoding issues in strftime() (pythonGH-12… …
a34caba 
…5193) (pythonGH-125657)

Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.

* Characters not encodable in the current locale are now acceptable in
  the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
  longer recombinated.
* Embedded null character no longer terminates the format string.

This fixes also pythongh-78662 and pythongh-124531.

(cherry picked from commit 08ccbb9)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit ad3eac1)
serhiy-storchaka added a commit that referenced this pull request Oct 17, 2024
@miss-islington @serhiy-storchaka
[3.12] gh-52551: Fix encoding issues in strftime() (GH-125193) (GH-12… …
232b303 
…5657) (GH-125661)

Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.

* Characters not encodable in the current locale are now acceptable in
  the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
  longer recombinated.
* Embedded null character no longer terminates the format string.

This fixes also gh-78662 and gh-124531.

(cherry picked from commit 08ccbb9)
(cherry picked from commit ad3eac1)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@vstinner

vstinner commented Oct 18, 2024

Copy link
Copy Markdown
Member

Congrats! That's a nice change. Using Unicode instead of bytes for the internal buffer is a good move.

ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
@serhiy-storchaka @ebonnal
pythongh-52551: Fix encoding issues in strftime() (pythonGH-125193) …
3a7c158 
Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.

* Characters not encodable in the current locale are now acceptable in
  the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
  longer recombinated.
* Embedded null character no longer terminates the format string.

This fixes also pythongh-78662 and pythongh-124531.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes

@pganssle pganssle Awaiting requested review from pganssle pganssle is a code owner

@abalkin abalkin Awaiting requested review from abalkin

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

datetime.strftime strings can be terminated by "\x00" literals time.strftime() and Unicode characters on Windows

2 participants

@serhiy-storchaka @vstinner