[update_lib] Apply test grouping to deps by youknowone · Pull Request #6867 · RustPython/RustPython

youknowone

📝 Walkthrough

Walkthrough

The PR refactors dependency resolution APIs to require explicit cpython_prefix and lib_prefix arguments instead of defaults, expands the DEPENDENCIES table with new module groups and test mappings, enhances hard-dependency tracking and resolution with auto-detection of _py module variants, adds multi-test-path support to the quick workflow, and introduces explicit UTF-8 encoding for file I/O operations.

Changes

Cohort / File(s)	Summary
Dependency Resolution Core `scripts/update_lib/deps.py`	Refactored all public/private function signatures to require explicit `cpython_prefix` and `lib_prefix` arguments (removed defaults). Enhanced `resolve_hard_dep_parent()` to auto-detect `_py{module}.py` and `_py_{module}.py` patterns and strip `.py` extensions. Expanded DEPENDENCIES with new module groups (__future__, site, opcode, pickle, re, weakref, etc.) and normalized test path entries by removing redundant "test/" prefixes. Added new public functions: `resolve_test_to_lib()` and `get_all_hard_deps()`. Updated all internal call sites to pass explicit prefixes.
File I/O Encoding `scripts/update_lib/migrate.py`	Added explicit UTF-8 encoding to all file read and write operations (`read_text(encoding="utf-8")` and `write_text(..., encoding="utf-8")`) across patching, copying, and content extraction flows. No signature or control flow changes.
Multi-Test Path Support `scripts/update_lib/quick.py`	Modified `git_commit()` signature to accept multiple test paths (single Path, list, or None) and normalized to list for staging. Updated `main()` to collect and process all test paths via `get_test_paths()` with fallback logic, iterating over each for the commit. Expanded `_expand_shortcut()` to include extension-module test shortcut (`test/test_<name>`).
Dependency Display & Resolution `scripts/update_lib/show_deps.py`	Removed default parameters from `get_all_modules()`, `format_deps()`, `show_deps()`, and related functions. Added `hard_deps` parameter to `format_deps_tree()` to display hard dependencies. Integrated `resolve_test_to_lib()` import and logic to resolve test-prefixed modules to library groups. Updated hard_deps collection to include explicit hard_deps and implicit ones derived from lib_paths.
Todo List & Tracking `scripts/update_lib/show_todo.py`	Removed default parameters from all public functions (`compute_todo_list()`, `get_all_tests()`, `get_untracked_files()`, etc.) requiring explicit `cpython_prefix` and `lib_prefix`. Added hard_deps tracking via `get_all_hard_deps()` and status checks via `is_up_to_date()`. Introduced new `get_original_files()` helper. Enhanced output formatting to display hard_deps status information (all deps in verbose mode; only outdated in normal mode).
Test Updates `scripts/update_lib/tests/test_deps.py`	Updated test assertions in `test_auto_detect_py_module` to use new module naming scheme (mymodule/_pymymodule instead of datetime/_pydatetime) and verify auto-detection of both `.py` and `_py{module}.py` patterns.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

RustPython/RustPython#6847 — Modifies overlapping dependency resolution codepaths (resolve_hard_dep_parent, test-module resolution) in deps.py and show_deps.py
RustPython/RustPython#6823 — Updates the same files (deps.py, show_todo.py) with cascading function signature changes and hard_deps tracking
RustPython/RustPython#6817 — Implements hard-dependency resolution and get_lib_paths auto-detection of _py module variants in deps.py

Suggested reviewers

moreal
ShaharNaveh

Poem

🐰 Hoppity-hop, no defaults stay!
Explicit paths light the way,
Hard deps now dance, tests align,
UTF-8 codes all sign,
Multi-path streams gently flow—
The updates bloom, watch them grow! 🌱✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Apply test grouping to deps' directly describes the main change: integrating test grouping into the dependency tracking system. It accurately reflects the PR's comprehensive refactoring of test resolution and dependency handling in deps.py.
Docstring Coverage	✅ Passed	Docstring coverage is 95.24% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}