◐ Shell
reader mode source ↗
Skip to content

gh-120754: Ensure _stat_atopen is cleared on fd change#125166

Merged
vstinner merged 11 commits into
python:mainfrom
cmaloney:cmaloney/fd_cleanup
Nov 1, 2024
Merged

gh-120754: Ensure _stat_atopen is cleared on fd change#125166
vstinner merged 11 commits into
python:mainfrom
cmaloney:cmaloney/fd_cleanup

Conversation

@cmaloney

@cmaloney cmaloney commented Oct 8, 2024

Copy link
Copy Markdown
Contributor

While adding _stat_atopen I accidentally created a leak in the C implementation because the _stat_atopen member was set to NULL in the C implementation, but the memory wasn't freed. That was resolved, but it showed there were a number of potential cases where _stat_atopen should be cleared (the information is out of date because the fd changed) that it was not. In this PR I audited by reading through FileIO (in _io and _pyio), and in all cases where the fd is changed/closed made sure to clear/reset _stat_atopen as well.

Followup from: #124225 (comment), cc: @vstinner

Performed an audit of `fileio.c` and `_pyio` and made sure anytime the
fd changes the stat result, if set, is also cleared/changed.

There's one case where it's not cleared, if code would clear it in
__init__, keep the memory allocated and just do another fstat with the
existing memory.

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

There are compiler warnings that you can see in the review tab.

@cmaloney

cmaloney commented Oct 9, 2024

Copy link
Copy Markdown
Contributor Author

I think the latest commit fixes the compiler warnings (was a testing/debugging assert I accidentally committed with a missing ;, latest changes remove that assert). Not able to find them in the latest changes.

@cmaloney

Copy link
Copy Markdown
Contributor Author

CI "Tests / Ubuntu SSL tests with OpenSSL" failed because of what looks like a network hang / not these changes: Received 164773440 of 168967744 (97.5%), 0.0 MBs/sec while doing "Configure ccache action"

@vstinner vstinner merged commit 72dd471 into python:main Nov 1, 2024
@vstinner

vstinner commented Nov 1, 2024

Copy link
Copy Markdown
Member

Merged, thank you.

@cmaloney cmaloney deleted the cmaloney/fd_cleanup branch November 1, 2024 21:55
picnixz pushed a commit to picnixz/cpython that referenced this pull request Dec 8, 2024
…n#125166)

Performed an audit of `fileio.c` and `_pyio` and made sure anytime the
fd changes the stat result, if set, is also cleared/changed.

There's one case where it's not cleared, if code would clear it in
__init__, keep the memory allocated and just do another fstat with the
existing memory.
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
…n#125166)

Performed an audit of `fileio.c` and `_pyio` and made sure anytime the
fd changes the stat result, if set, is also cleared/changed.

There's one case where it's not cleared, if code would clear it in
__init__, keep the memory allocated and just do another fstat with the
existing memory.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants