◐ Shell
reader mode source ↗
Skip to content

windows umask, win32_xstat_slow_impl, fake EXT_SUFFIX#6340

Merged
youknowone merged 4 commits into
RustPython:mainfrom
youknowone:nt
Dec 8, 2025
Merged

windows umask, win32_xstat_slow_impl, fake EXT_SUFFIX#6340
youknowone merged 4 commits into
RustPython:mainfrom
youknowone:nt

Conversation

@youknowone

@youknowone youknowone commented Dec 7, 2025

Copy link
Copy Markdown
Member

Summary by CodeRabbit

  • New Features

    • Added umask() support on Windows for setting process file mode creation mask.
  • Improvements

    • Major overhaul of Windows file metadata/stat handling for more accurate file types, symlink/reparse handling, and fewer edge-case errors.
    • Updated system configuration defaults to include an EXT_SUFFIX of ".pyd" and expose SOABI.
  • Chores

    • Added SOABI token to the spelling/dictionary list.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai

coderabbitai Bot commented Dec 7, 2025

Copy link
Copy Markdown
Contributor

Walkthrough

Adds Windows-specific stat/xstat implementation and helpers, exposes a winsock initializer, adds an FFI-backed umask wrapper in the nt stdlib module, extends sysconfig config vars (EXT_SUFFIX, SOABI, Py_DEBUG), and inserts the token SOABI into the CPython dictionary file.

Changes

Cohort / File(s) Summary
Dictionary Updates
.cspell.dict/cpython.txt
Added token SOABI to the CPython word list
Stdlib: nt & sysconfig
crates/vm/src/stdlib/nt.rs, crates/vm/src/stdlib/sysconfig.rs
Added extern C _umask declaration and a #[pyfunction] umask(mask: i32) -> i32 wrapper; extended config_vars with EXT_SUFFIX (".pyd"), SOABI (None), and Py_DEBUG (false) entries
Windows file metadata & stat/xstat
crates/vm/src/windows.rs
Introduced comprehensive Windows stat/xstat slow-path implementation, new file-attribute and mode constants, reparse-point/symlink handling, FILETIME/time conversions, inode/file-id logic, attributes_from_dir, attribute_data_to_stat, and pub fn init_winsock()

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

  • Focus review on Windows API usage, reparse-point and symlink handling, and FILETIME-to-epoch conversions in crates/vm/src/windows.rs.
  • Verify safety and correct linkage for the new extern _umask and the umask wrapper in crates/vm/src/stdlib/nt.rs.
  • Confirm EXT_SUFFIX hardcoding and other config choices in crates/vm/src/stdlib/sysconfig.rs align with intended packaging behavior.

Possibly related PRs

Poem

🐰 A tiny hop through files and time,
I add a mask and parse a rhyme,
SOABI tucked into the list,
Windows stats now gently kissed,
Hooray—this rabbit leaves a chime! 🎋

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly references three key changes in the changeset: windows umask (new FFI binding in nt.rs), win32_xstat_slow_impl (new Windows stat implementation in windows.rs), and fake EXT_SUFFIX (sysconfig.rs initialization).
Docstring Coverage ✅ Passed Docstring coverage is 90.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@youknowone youknowone force-pushed the nt branch 2 times, most recently from 8dcbecb to 0c70492 Compare December 7, 2025 16:56
@youknowone youknowone marked this pull request as ready for review December 7, 2025 16:56

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Actionable comments posted: 3

🧹 Nitpick comments (2)
crates/vm/src/windows.rs (2)

400-408: Type mismatch: using local struct with windows_sys enum constant.

Line 405 uses FileAttributeTagInfo which is imported as a FILE_INFO_BY_HANDLE_CLASS enum variant from windows_sys (line 283), but you're using it as both the enum value (line 405) and casting to a local struct type (line 401, 406). This works because the enum value happens to match the expected constant, but the struct being zeroed is the local FileAttributeTagInfo, not the windows_sys structure.

This is fragile and confusing. Consider:

+// Use explicit constant for clarity
+const FILE_ATTRIBUTE_TAG_INFO_CLASS: i32 = 9; // FileAttributeTagInfo enum value

 let ret = unsafe {
     GetFileInformationByHandleEx(
         h_file,
-        FileAttributeTagInfo,
+        FILE_ATTRIBUTE_TAG_INFO_CLASS,
         &mut local_tag_info as *mut _ as *mut _,
         std::mem::size_of::<FileAttributeTagInfo>() as u32,
     )
 };

Or rename the local struct to avoid the collision entirely.


370-371: Consider RAII wrapper for handle cleanup.

The closure pattern for ensuring handle cleanup works, but it's somewhat unusual in Rust. A more idiomatic approach would be to use an RAII wrapper that automatically closes the handle on drop:

struct OwnedHandle(RAW_HANDLE);
impl Drop for OwnedHandle {
    fn drop(&mut self) {
        if self.0 != INVALID_HANDLE_VALUE {
            unsafe { CloseHandle(self.0) };
        }
    }
}

This is optional since the current approach is correct; it's just a style suggestion for future maintenance.

Also applies to: 493-500

📜 Review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cab41c8 and 0c70492.

⛔ Files ignored due to path filters (4)
  • Lib/test/test_genericpath.py is excluded by !Lib/**
  • Lib/test/test_logging.py is excluded by !Lib/**
  • Lib/test/test_ntpath.py is excluded by !Lib/**
  • Lib/test/test_shutil.py is excluded by !Lib/**
📒 Files selected for processing (4)
  • .cspell.dict/cpython.txt (1 hunks)
  • crates/vm/src/stdlib/nt.rs (1 hunks)
  • crates/vm/src/stdlib/sysconfig.rs (1 hunks)
  • crates/vm/src/windows.rs (3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style by running cargo fmt to format Rust code
Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints introduced by changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

  • crates/vm/src/stdlib/nt.rs
  • crates/vm/src/windows.rs
  • crates/vm/src/stdlib/sysconfig.rs
🧠 Learnings (4)
📓 Common learnings
Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration using #ifdef checks rather than providing fallback values for other platforms.
Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration rather than providing fallback values for other platforms.
📚 Learning: 2025-11-29T12:17:28.606Z
Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-29T12:17:28.606Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust

Applied to files:

  • crates/vm/src/stdlib/nt.rs
📚 Learning: 2025-06-27T14:47:28.810Z
Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration rather than providing fallback values for other platforms.

Applied to files:

  • crates/vm/src/stdlib/nt.rs
  • crates/vm/src/stdlib/sysconfig.rs
📚 Learning: 2025-06-27T14:47:28.810Z
Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration using #ifdef checks rather than providing fallback values for other platforms.

Applied to files:

  • crates/vm/src/stdlib/nt.rs
  • crates/vm/src/stdlib/sysconfig.rs
🧬 Code graph analysis (2)
crates/vm/src/stdlib/nt.rs (1)
crates/vm/src/stdlib/posix.rs (1)
  • umask (1328-1330)
crates/vm/src/windows.rs (1)
crates/vm/src/stdlib/winapi.rs (2)
  • CloseHandle (83-85)
  • GetFileType (163-173)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Run rust tests (ubuntu-latest)
  • GitHub Check: Run rust tests (macos-latest)
  • GitHub Check: Check Rust code with clippy
  • GitHub Check: Ensure compilation on various targets
  • GitHub Check: Check the WASM package and demo
🔇 Additional comments (5)
.cspell.dict/cpython.txt (1)

49-49: LGTM!

The SOABI token is correctly added to the dictionary in alphabetical order. This aligns with the new SOABI config variable introduced in sysconfig.rs.

crates/vm/src/windows.rs (4)

113-130: LGTM!

The attributes_to_mode function correctly ports CPython's logic for converting Windows file attributes to Unix-style mode bits.


132-227: LGTM!

The attribute_data_to_stat function is a comprehensive port of CPython's _Py_attribute_data_to_stat. The time conversion logic correctly handles the Windows-to-Unix epoch difference, and the 128-bit file ID support via st_ino_high is properly implemented.


229-267: LGTM!

The attributes_from_dir fallback correctly uses FindFirstFileW when CreateFileW fails due to access/sharing issues. The handle cleanup is properly handled.


534-535: LGTM!

The fallback to win32_xstat_slow_impl is correctly integrated, providing robust handling when the fast path (GetFileInformationByName) cannot be used.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Actionable comments posted: 1

♻️ Duplicate comments (1)
crates/vm/src/windows.rs (1)

105-111: FileAttributeTagInfo naming is slightly confusing but not incorrect.

Defining a local struct FileAttributeTagInfo while also importing the FileAttributeTagInfo enum value from windows_sys is valid (type vs value namespace), but it’s easy to misread. Consider a small rename of the struct (e.g. LocalFileAttributeTagInfo) to make the distinction clearer for future readers.

Also applies to: 400-408

🧹 Nitpick comments (2)
crates/vm/src/stdlib/sysconfig.rs (1)

11-21: EXT_SUFFIX stub is knowingly wrong; please follow up with a platform-aware implementation.

The .pyd value and SOABI = None make _sysconfig diverge from CPython on all non‑Windows platforms (and even on Windows, it’s missing ABI tagging). Since the FIXME already calls this out and the PR title mentions “fake EXT_SUFFIX”, it’s fine as a temporary stub, but this will need a follow‑up that:

  • Computes EXT_SUFFIX conditionally per target (Windows vs Unix), and
  • Keeps EXT_SUFFIX and SOABI consistent (e.g. EXT_SUFFIX derived from SOABI).

Classifying as a reminder rather than a blocker for this PR.

crates/vm/src/windows.rs (1)

231-267: attributes_from_dir fallback is correct but could benefit from a brief comment on limitations.

The FindFirstFileW-based fallback populates a minimal BY_HANDLE_FILE_INFORMATION and reparse tag when CreateFileW fails for access/sharing reasons, which matches CPython’s approach.

A tiny nit: a short comment noting that volume serial / file index aren’t available via this path (and will keep their zeroed defaults) would clarify why inode/dev fields may differ vs the direct handle-based path.

📜 Review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0c70492 and 8317e6d.

⛔ Files ignored due to path filters (4)
  • Lib/test/test_genericpath.py is excluded by !Lib/**
  • Lib/test/test_logging.py is excluded by !Lib/**
  • Lib/test/test_ntpath.py is excluded by !Lib/**
  • Lib/test/test_shutil.py is excluded by !Lib/**
📒 Files selected for processing (4)
  • .cspell.dict/cpython.txt (1 hunks)
  • crates/vm/src/stdlib/nt.rs (1 hunks)
  • crates/vm/src/stdlib/sysconfig.rs (1 hunks)
  • crates/vm/src/windows.rs (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • crates/vm/src/stdlib/nt.rs
  • .cspell.dict/cpython.txt
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style by running cargo fmt to format Rust code
Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints introduced by changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

  • crates/vm/src/stdlib/sysconfig.rs
  • crates/vm/src/windows.rs
🧠 Learnings (4)
📓 Common learnings
Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration using #ifdef checks rather than providing fallback values for other platforms.
Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration rather than providing fallback values for other platforms.
📚 Learning: 2025-06-27T14:47:28.810Z
Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration using #ifdef checks rather than providing fallback values for other platforms.

Applied to files:

  • crates/vm/src/stdlib/sysconfig.rs
📚 Learning: 2025-06-27T14:47:28.810Z
Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration rather than providing fallback values for other platforms.

Applied to files:

  • crates/vm/src/stdlib/sysconfig.rs
📚 Learning: 2025-06-28T16:31:03.991Z
Learnt from: arihant2math
Repo: RustPython/RustPython PR: 5790
File: build.rs:2-2
Timestamp: 2025-06-28T16:31:03.991Z
Learning: In Cargo build scripts (build.rs), the environment variable CARGO_CFG_TARGET_OS is guaranteed to exist and is automatically set by Cargo during the build process, making unwrap() safe to use when accessing this variable.

Applied to files:

  • crates/vm/src/stdlib/sysconfig.rs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
  • GitHub Check: Run snippets and cpython tests on wasm-wasi
  • GitHub Check: Check the WASM package and demo
  • GitHub Check: Check Rust code with clippy
  • GitHub Check: Run snippets and cpython tests (windows-latest)
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
  • GitHub Check: Ensure compilation on various targets
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Run rust tests (macos-latest)
  • GitHub Check: Run rust tests (ubuntu-latest)
🔇 Additional comments (2)
crates/vm/src/windows.rs (2)

96-130: Mode mapping from Windows attributes looks correct and matches CPython semantics.

The new constants and attributes_to_mode() logic (directory vs regular file, read‑only vs writable, execute bits for directories) align with CPython’s attributes_to_mode behavior. This is a solid foundation for the rest of the stat translation.


270-501: win32_xstat_slow_impl and fallback wiring in win32_xstat_impl look faithful to CPython.

  • Error handling around CreateFileW mirrors CPython’s special cases (access denied/sharing violation → directory fallback, invalid parameter → retry with GENERIC_READ, ERROR_CANT_ACCESS_FILE with traverse → unhandled reparse handling).
  • Non‑disk handles (char devices, pipes, unknown type) are mapped to appropriate st_mode bits and short‑circuit with a defaulted StatStruct.
  • Reparse tag probing with GetFileInformationByHandleEx(FileAttributeTagInfo, ...) plus the is_unhandled_tag logic and recursive traversal for non‑link reparse points matches the CPython control flow.
  • Optional FILE_BASIC_INFO and FILE_ID_INFO are used when available; failures downgrade gracefully.
  • Handle lifetime is correctly managed: all successful open paths are closed once, and recursive win32_xstat_slow_impl(path, true) calls close h_file before recursing, avoiding leaks/double‑closes.

Overall this is a strong, careful port; I don’t see functional bugs in the flow.

It’d be worth running the existing CPython Windows stat/xstat test suite (or RustPython’s equivalent) against this implementation on real Windows targets, particularly cases involving:

  • Junctions and non‑symlink reparse points,
  • Unhandled tags with traverse=True/False,
  • Special paths like \\.\CON and named pipes.

Hide details View details @youknowone youknowone merged commit 876368e into RustPython:main Dec 8, 2025
13 checks passed
@youknowone youknowone deleted the nt branch December 8, 2025 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant