automark add reasons to reason-less expectedFailures and handles parent better by youknowone · Pull Request #6936 · RustPython/RustPython

youknowone

📝 Walkthrough

Walkthrough

This PR enhances test failure marking with inheritance-aware handling, allowing parent class test markers to be set when all child inheritors fail, and enabling stripping of unreasoned failure decorators. It includes a minor formatting adjustment to a Rust debug assertion and comprehensive test updates.

Changes

Cohort / File(s)	Summary
Formatting Update `crates/codegen/src/ir.rs`	Reformatted a debug_assert! macro call string across multiple lines for improved readability without changing logic or behavior.
Inheritance-Aware Test Failure Handling `scripts/update_lib/cmd_auto_mark.py`	Added five new helper functions (`_expand_stripped_to_children`, `_consolidate_to_parent`, `_method_removal_range`, `_find_all_inheritors`, `strip_reasonless_expected_failures`) to handle inheritance-based test failure consolidation. Modified control flow in `auto_mark_file` and `auto_mark_directory` to strip reasonless decorators, re-expand to inheritors, and consolidate failures to parent classes when all children fail. Enhanced `remove_expected_failures` to account for decorators and comments.
Test Suite Expansion `scripts/update_lib/tests/test_auto_mark.py`	Rewrote and significantly expanded test suite with new test cases covering inheritance scenarios, failure consolidation, reasonless decorator stripping, failure expansion to child classes, and smart auto-mark filtering logic. Added helper `_make_result` for creating test fixtures.

Sequence Diagram

sequenceDiagram
    participant FileSystem as File System
    participant AutoMark as Auto-Mark Engine
    participant CodeAnalysis as Code Analysis
    participant TestRunner as Test Runner
    participant Consolidator as Consolidator

    FileSystem->>AutoMark: Load test file
    AutoMark->>AutoMark: Strip reasonless `@expectedFailure` decorators
    AutoMark->>FileSystem: Write modified file
    AutoMark->>TestRunner: Run tests
    TestRunner->>TestRunner: Collect failures (parent & child classes)
    TestRunner->>AutoMark: Return failing tests
    AutoMark->>CodeAnalysis: Expand stripped markers to inheritors
    CodeAnalysis->>CodeAnalysis: Find all classes inheriting methods
    CodeAnalysis->>AutoMark: Return expanded failure set
    AutoMark->>Consolidator: Attempt consolidation
    Consolidator->>Consolidator: Check if all inheritors failed
    Consolidator->>Consolidator: Consolidate to parent class
    Consolidator->>AutoMark: Return consolidated failures
    AutoMark->>FileSystem: Apply `@expectedFailure` markers

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

Improve deps output #6874: Modifies the same scripts/update_lib/cmd_auto_mark.py module with direct code-level changes to auto-marking logic.
auto mark parent tests #6778: Adds inheritance-aware auto-marking logic with class hierarchy resolution and parent consolidation functionality.
rename fix_test and support removing unexpected success #6748: Modifies expected-failure removal and super-call detection logic that overlaps with removal range computation.

Suggested reviewers

moreal
ShaharNaveh

Poem

🐰 Hops through class hierarchies with glee,
Parent and child, now in harmony!
Strip the unmarked, expand the bound,
Consolidate failures all around,
Test inheritance, safe and sound! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 63.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the main objectives: adding reasons to reason-less expectedFailures and improving parent class handling for auto-marking test failures.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}