update_lib hard dependency resolver#6817

youknowone

updaing datetime will triggers to update also _pydatetime

cc @moreal @ShaharNaveh

Summary by CodeRabbit

New Features
- Enhanced dependency resolution for library updates, automatically detecting and processing test-related dependencies and data paths.
Bug Fixes
- Improved test failure reporting with clearer error messages.
- Fixed handling of package initialization files in test path detection.
- Enhanced library copying to support complex package structures.
Tests
- Added comprehensive test coverage for dependency resolution and library operations.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai

📝 Walkthrough

Walkthrough

This PR extends the library update toolchain with a new dependency resolution module (deps.py), improves error handling with TestRunError exception, refactors multi-file copying support via a helper function, and adjusts path resolution for __init__.py files. Comprehensive tests are added across all modified modules.

Changes

Cohort / File(s)	Summary
Documentation & Configuration `\.claude/commands/upgrade-pylib\.md`	Minor indentation adjustment in the STOP section for formatting alignment.
Error Handling & Test Validation `scripts/update_lib/auto_mark\.py`	Introduces `TestRunError` exception class. Replaces `capture_output` with `stdout=PIPE` and `stderr=None` for selective output handling. Adds validation in `_is_super_call_only` to match method names. Raises `TestRunError` when test runs produce no results.
Multi-File Copy Support `scripts/update_lib/copy_lib\.py`	Introduces private helper `_copy_single()` for single file/directory copies. Refactors `copy_lib()` to detect `/Lib/` paths and copy multiple files via `get_lib_paths()`, with fallback to single-file mode.
Dependency Resolution (New Module) `scripts/update_lib/deps\.py`	New module implementing dependency resolution with functions: `get_lib_paths()`, `get_test_paths()`, `get_data_paths()`, `parse_test_imports()`, `get_test_dependencies()`, and `resolve_all_paths()`. Includes `DEPENDENCIES` and `TEST_DEPENDENCIES` mappings for irregular library layouts and test-specific path overrides.
Import Insertion `scripts/update_lib/patch_spec\.py`	Replaces unconditional assertion with guarded fallback logic: returns last import line if present, otherwise returns docstring end line, else returns 0.
Path Resolution `scripts/update_lib/path\.py`	Adjusts `lib_to_test_path()` to use parent directory name when encountering `__init__.py` instead of the literal filename.
Workflow Enhancement `scripts/update_lib/quick\.py`	Adds Step 1.5 to process test dependencies after initial patch via `get_test_dependencies()`. Introduces `DEPENDENCIES`-based overrides in `_expand_shortcut()`. Adds exception handling for `TestRunError` with clean error output.
Test Suite (New & Updated) `scripts/update_lib/tests/test_auto_mark\.py`, `test_copy_lib\.py`, `test_deps\.py`, `test_patch_spec\.py`, `test_path\.py`, `test_quick\.py`	Comprehensive unit tests added: validation of `_is_super_call_only`, `_copy_single` file/directory operations, dependency resolution across all path types, import insertion fallback logic, `__init__.py` parent name handling, shortcut overrides, and test failure scenarios.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Remake update_lib #6796: Continuation of the same update\_lib toolchain development, modifying identical modules (auto\_mark.py, copy\_lib.py, patch\_spec.py, path.py, quick.py) and introducing the new deps.py and test infrastructure.

Poem

🐰 With paths untangled and tests that stand firm,
Dependencies mapped—our toolchain does turn!
From __init__.py secrets to copies that scale,
This rabbit hops faster, leaves no library pale. ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'update_lib hard dependency resolver' accurately captures the main objective of this PR, which is to implement dependency resolution so that updating datetime triggers _pydatetime updates.
Docstring Coverage	✅ Passed	Docstring coverage is 96.49% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions

Code has been automatically formatted

The code in this PR has been formatted using:

ruff format
Please pull the latest changes before pushing again:

git pull origin lib_updater

ShaharNaveh

I like this change a lot!

coderabbitai

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@scripts/update_lib/copy_lib.py`:
- Around line 65-77: The code that parses src_path via path_str and splits on
"/Lib/" mis-handles nested module paths and makes the "else" fallback
unreachable because parse_lib_path always raises; update the logic in
copy_lib.py so that when "/Lib/" is present you preserve the full after_lib (not
just its first path component) and derive the destination accordingly (e.g., use
the full nested path portion from after_lib to compute lib_path), and when
"/Lib/" is not present call parse_lib_path inside a try/except and on exception
perform an explicit direct copy using _copy_single (or consult DEPENDENCIES
mapping) instead of assuming parse_lib_path will succeed; refer to
variables/path pieces path_str, cpython_prefix, after_lib, name, and functions
parse_lib_path and _copy_single to locate the changes.

In `@scripts/update_lib/deps.py`:
- Around line 358-365: In resolve_all_paths, the loop over deps returned from
get_test_dependencies is appending dictionary keys ("hard_deps"/"data") into
result["test_deps"]; change the logic so you iterate over deps["hard_deps"]
(and, if you need test data, also extend/merge deps["data"]) instead of
iterating the deps dict itself; reference get_test_dependencies, the local
variable deps, and result["test_deps"], and add existence checks for "hard_deps"
and "data" before iterating to avoid KeyError.

🧹 Nitpick comments (2)

scripts/update_lib/tests/test_patch_spec.py (1)
325-358: Good test coverage for the three main branches.

The tests correctly validate each branch of _find_import_insert_line. Consider adding a test for a multi-line docstring to verify end_lineno handling, though this is optional since the current coverage is sufficient.
🧪 Optional: Additional test for multi-line docstring
def test_no_imports_with_multiline_docstring(self):
    """Test fallback to after multi-line docstring when no imports."""
    code = '''"""Module docstring.

This spans multiple lines.
"""

class Foo:
    pass
'''
    tree = ast.parse(code)
    line = _find_import_insert_line(tree)
    self.assertEqual(line, 4)
scripts/update_lib/quick.py (1)
108-133: Dependency migration is solid; please confirm data deps are directories.
shutil.copytree will fail if any entry in test_deps["data"] is a file. If files are possible, consider handling both cases.
Optional hardening for file vs directory data deps
-            if data_lib.exists():
-                shutil.rmtree(data_lib)
-            shutil.copytree(data_src, data_lib)
+            if data_lib.exists():
+                if data_lib.is_dir():
+                    shutil.rmtree(data_lib)
+                else:
+                    data_lib.unlink()
+            if data_src.is_dir():
+                shutil.copytree(data_src, data_lib)
+            else:
+                data_lib.parent.mkdir(parents=True, exist_ok=True)
+                shutil.copy2(data_src, data_lib)

youknowone added 2 commits January 20, 2026 21:41

update_lib __init__ handling

7bd50b2

dependency

9b7b53c

youknowone added the skip:ci Skip running the ci label Jan 20, 2026

Auto-format: ruff format

24c1b88

ShaharNaveh reviewed Jan 20, 2026

View reviewed changes

ShaharNaveh approved these changes Jan 20, 2026

View reviewed changes

youknowone marked this pull request as ready for review January 20, 2026 14:15

coderabbitai Bot reviewed Jan 20, 2026

View reviewed changes

youknowone deleted the lib_updater branch January 20, 2026 14:52

This was referenced Jan 30, 2026

[update_lib] Fix async func auto-mark #6903

Merged

Make auto-mark output deterministic and fix blank line leak #6957

Merged

coderabbitai Bot mentioned this pull request Feb 9, 2026

[update_lib] fix hard_deps resolution and fix commit not to miss test data #7058

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update_lib hard dependency resolver#6817