◐ Shell
clean mode source ↗

Replace `compiler::source` module with `ruff_source_file` by ShaharNaveh · Pull Request #6016 · RustPython/RustPython

ShaharNaveh

compiler/Cargo.toml (1)

23-25: Good move to inherit lints from workspace

Forwarding the lint configuration (workspace = true) removes duplication and ensures compilation uses the central policy consistently. No issues spotted.

vm/src/frame.rs (1)

24-24: LGTM - Import updated correctly for the source module refactor.

The import change from crate::source::SourceLocation to ruff_source_file::SourceLocation correctly implements the intended refactor to use the upstream ruff_source_file crate instead of the custom compiler::source module.

vm/src/builtins/code.rs (2)

18-18: LGTM - Import correctly updated to use upstream type.

The import change from a custom LineNumber type to OneIndexed from ruff_source_file aligns with the PR's objective to use upstream source handling types.

359-359: LGTM - Constructor usage correctly updated.

The change from LineNumber::new to OneIndexed::new correctly implements the migration to the upstream type while maintaining the same 1-indexed line number semantics.

vm/src/compiler.rs (1)

3-3: LGTM - Correctly removes obsolete source re-export.

The removal of the rustpython_compiler_source re-export as source is consistent with the PR's objective to eliminate the custom source module entirely in favor of ruff_source_file. Since the compiler/source crate has been removed, this re-export is no longer needed.

vm/Cargo.toml (2)

23-23: LGTM: AST feature dependency update is correct.

Removing ruff_source_file from the "ast" feature dependencies makes sense since it's now a mandatory dependency rather than optional.

39-39: LGTM: Dependency change aligns with refactoring goals.

Converting ruff_source_file from an optional to mandatory dependency is correct for this refactoring, as SourceFile is now the unified source representation throughout the codebase.

vm/src/stdlib/ast/elif_else_clause.rs (4)

2-2: LGTM: Import statement correctly updated.

The import change from rustpython-compiler-source to ruff_source_file::SourceFile aligns with the refactoring objectives.

4-9: LGTM: Function signature correctly updated.

The parameter change from source_code: &SourceCodeOwned to source_file: &SourceFile is consistent with the overall refactoring to use ruff_source_file types.

13-38: LGTM: All method calls properly updated.

All calls to ast_to_object, ast_from_object, and node_add_location correctly use the new source_file parameter. The logic remains unchanged.

42-54: LGTM: ast_from_object function consistently updated.

The function signature and all internal calls properly use source_file instead of source_code, maintaining consistency with the refactoring.

vm/src/stdlib/ast/basic.rs (4)

1-1: LGTM: Import correctly updated for the refactoring.

The import change to ruff_source_file::SourceFile is consistent with replacing the custom source handling.

6-20: LGTM: Identifier Node implementation properly updated.

Both ast_to_object and ast_from_object methods correctly use the new SourceFile parameter type. Using _source_file indicates it's unused for this basic type, which is appropriate.

22-37: LGTM: Int Node implementation consistently updated.

The method signatures are properly updated to use SourceFile, maintaining consistency with the refactoring goals.

39-51: LGTM: Bool Node implementation correctly updated.

Both trait methods properly use the new SourceFile parameter type, completing the consistent update across all basic types.

vm/src/stdlib/ast/node.rs (6)

2-2: LGTM: Import statement correctly updated.

The import change to ruff_source_file::SourceFile aligns with the refactoring to use the upstream source handling.

4-16: LGTM: Node trait definition properly updated.

The trait methods correctly use source_file: &SourceFile parameter, establishing the new interface for all AST node conversions throughout the system.

18-36: LGTM: Vec implementation consistently updated.

Both ast_to_object and ast_from_object methods properly use the new SourceFile parameter and correctly propagate it to the underlying node conversions.

38-54: LGTM: Box implementation correctly updated.

The implementation properly delegates to the underlying type while maintaining the new parameter signature.

56-76: LGTM: Option implementation properly updated.

The optional node handling correctly uses the new SourceFile parameter for both conversion directions.

80-94: LGTM: BoxedSlice implementation consistently updated.

The implementation correctly converts to/from Vec and properly uses the new parameter signature.

compiler/codegen/src/unparse.rs (4)

2-2: LGTM: Import statement correctly updated.

The import change to ruff_source_file::SourceFile aligns with the refactoring goals.

30-38: LGTM: Unparser struct and constructor properly updated.

The struct field and constructor signature correctly use SourceFile instead of the previous SourceCode type.

533-533: LGTM: Method call updated for new SourceFile API.

The change from get_range(val.range()) to slice(val.range()) reflects the different API of SourceFile compared to the previous SourceCode type. This appears to be the correct method for extracting source text ranges.

605-612: LGTM: UnparseExpr struct and function properly updated.

Both the struct field and the unparse_expr function signature correctly use SourceFile, maintaining consistency with the refactoring.

vm/src/stdlib/ast/operator.rs (1)

2-3: Migration looks correct — no functional or stylistic issues spotted

All Node impls were updated to accept &SourceFile, the unused parameter is correctly prefixed with an underscore to silence clippy, and no other logic changed.
Looks good.

Also applies to: 6-20, 38-63, 101-119, 139-163

vm/src/builtins/traceback.rs (1)

6-18: 👍 Clean switch to OneIndexed; validation is explicit

Importing ruff_source_file::OneIndexed and storing it directly in the struct eliminates the previous alias and enforces 1-based invariants. py_new now checks lineno > 0, so runtime safety is preserved.
No further action needed.

vm/src/stdlib/ast/argument.rs (1)

10-13: Consistent use of source_file propagated correctly

PositionalArguments and KeywordArguments now forward source_file to inner calls, ensuring location information isn’t lost after the refactor. No further issues detected.

Also applies to: 34-38

vm/src/stdlib/ast/other.rs (1)

65-81: Alias conversion updated correctly – thanks for adding node_add_location

Conversion now passes source_file to every nested node and retains range info; implementation looks solid.

vm/src/stdlib/ast/module.rs (2)

28-35: Top-level enum delegates updated – no issues

Each variant forwards the new source_file parameter; matches trait changes elsewhere.

65-90: ModModule::ast_to_object preserves location correctly after refactor

node_add_location now receives source_file, ensuring correct line/column mapping. Implementation is spot-on.

vm/src/stdlib/ast/statement.rs (1)

1-1237: LGTM! Systematic parameter type refactor completed correctly.

This refactor successfully replaces SourceCodeOwned with SourceFile across all AST statement node implementations. The changes are:

  • Consistent parameter type updates in all Node trait methods
  • Proper import addition for ruff_source_file::SourceFile
  • Systematic variable name updates from source_code to source_file
  • All helper function calls (node_add_location, range_from_object) correctly updated

The refactor maintains all existing logic and error handling while unifying the source representation type across the codebase.

vm/src/stdlib/ast/type_ignore.rs (1)

1-75: LGTM! Consistent parameter type refactor.

The changes correctly update all Node trait implementations to use SourceFile instead of SourceCodeOwned, maintaining consistency with the broader codebase refactor. All method signatures and helper function calls are properly updated.

vm/src/stdlib/ast/type_parameters.rs (1)

1-206: LGTM! Type parameter nodes consistently refactored.

All type parameter AST node implementations have been properly updated to use SourceFile instead of SourceCodeOwned. The refactor is systematic and maintains all existing functionality while aligning with the broader source representation unification.

vm/src/stdlib/ast/parameter.rs (1)

1-406: LGTM! Parameter handling consistently refactored.

The refactor successfully updates all parameter-related AST nodes and helper structures to use SourceFile. The complex parameter default extraction and merging logic is preserved while the source representation type is updated throughout. All method signatures and internal calls are properly updated.

vm/src/stdlib/ast/exception.rs (1)

1-81: LGTM! Exception handler nodes consistently refactored.

The exception handler AST node implementations are properly updated to use SourceFile instead of SourceCodeOwned. All method signatures, helper function calls, and error handling are correctly maintained while aligning with the source representation refactor.

vm/src/stdlib/ast/string.rs (1)

216-353: LGTM: Systematic parameter type migration executed correctly.

The migration from SourceCodeOwned to SourceFile has been implemented consistently across all Node trait implementations:

  • All method signatures properly updated from source_code: &SourceCodeOwned to source_file: &SourceFile
  • All internal calls to helper functions (node_add_location, range_from_object, ast_to_object, ast_from_object) correctly updated to use source_file
  • Logic and control flow remain unchanged, maintaining correctness

This change aligns with the broader refactor to replace the custom source representation with ruff_source_file::SourceFile.

vm/src/stdlib/ast/expression.rs (3)

2-7: Import organization and SourceFile integration look good.

The import changes properly support the migration:

  • Added use ruff_source_file::SourceFile; for the new parameter type
  • Reorganized related imports for better grouping and clarity

11-142: Comprehensive parameter migration in main expression dispatch.

The central ruff::Expr implementation has been systematically updated:

  • All ast_to_object calls correctly pass source_file instead of source_code
  • The ast_from_object method signature properly updated to use &SourceFile
  • All recursive calls to individual expression types updated consistently
  • Logic flow and match patterns remain unchanged, preserving correctness

146-1238: All expression node implementations consistently updated.

Every individual expression node type (ExprBoolOp, ExprNamed, ExprBinOp, etc.) has been properly migrated:

  • Method signatures updated from source_code: &SourceCodeOwned to source_file: &SourceFile
  • All internal calls to helper functions (node_add_location, range_from_object, get_node_field, etc.) correctly use source_file
  • Recursive calls to nested AST nodes properly pass the source_file parameter
  • No changes to the underlying logic, maintaining semantic correctness

This comprehensive update ensures all expression types work with the new SourceFile representation.

compiler/codegen/src/symboltable.rs (8)

21-21: Import update looks correct.

The import change from the previous source types to SourceFile and SourceLocation from ruff_source_file is appropriate and necessary for the migration.

78-88: Function signature updates are correct.

The changes to scan_program and scan_expr to accept SourceFile instead of the previous source type are consistent with the migration and properly pass the parameter to SymbolTableBuilder::new.

90-93: Method relocation is appropriate.

Moving the lookup method into the main impl SymbolTable block is a good organizational change that doesn't affect functionality.

608-616: Struct refactoring improves design.

The removal of the lifetime parameter from SymbolTableBuilder and storing SourceFile directly is an excellent improvement. This simplifies the type signature and is efficient since SourceFile uses Arc internally, making it cheap to clone.

Also applies to: 631-642

699-703: Method adaptation is correct.

The call to source_file.to_source_code().line_index() properly adapts to the new SourceFile API while maintaining compatibility with existing line index logic.

1050-1053: Error location handling updated correctly.

The adaptation to use source_file.to_source_code().source_location() for error reporting maintains proper location information in the new API.

1290-1290: Error location mapping is correct.

The change to use source_file.to_source_code().source_location() for named expression error reporting is appropriate.

1570-1573: Source location extraction updated properly.

The adaptation to extract source location using source_file.to_source_code().source_location() in the register_name method maintains correct location tracking.

compiler/src/lib.rs (5)

1-1: Import update is correct.

The import of SourceFile, SourceFileBuilder, and SourceLocation from ruff_source_file properly replaces the previous source handling types.

46-56: Error construction properly updated.

The from_ruff_parse_error method correctly adapts to use SourceFile methods (to_source_code().source_location() and name()) for error location and path information.

99-104: Source file creation is appropriate.

The use of SourceFileBuilder::new(source_path, source).finish() to create a SourceFile instance is the correct pattern for the new API.

122-138: Internal compile function updated correctly.

The function signature change to accept SourceFile and the usage of source_file.source_text() for parsing are consistent with the migration to the new source representation.

145-172: Symbol table compilation functions updated properly.

Both compile_symtable and _compile_symtable are correctly updated to use SourceFile creation and API calls. The error conversion at the end properly uses source_file.name() for the error context.

vm/src/stdlib/ast.rs (7)

24-24: Import update is comprehensive and correct.

The import of LineIndex, OneIndexed, SourceFile, SourceFileBuilder, and SourceLocation from ruff_source_file properly replaces the previous source handling imports and includes all necessary types.

127-159: Source range conversion properly updated.

The text_range_to_source_range function correctly adapts to use SourceFile methods (clone().source_text() and source_text()) to access the source content and maintain line indexing functionality.

161-184: Range parsing function updated correctly.

The range_from_object function properly accepts a &SourceFile parameter and passes it to source_range_to_text_range, maintaining the API consistency.

186-206: Source-to-text range conversion updated appropriately.

The source_range_to_text_range function correctly uses the new SourceFile API with clone().source_text() and source_text() calls for line indexing operations.

208-231: Location addition function updated correctly.

The node_add_location function properly accepts a &SourceFile parameter and uses it with text_range_to_source_range to maintain location information in AST nodes.

234-255: Parse function migration is correct.

The parse function properly creates a SourceFile using SourceFileBuilder and uses it throughout the parsing pipeline, including error handling and AST object creation.

258-286: Compile function updated appropriately.

The compile function correctly creates SourceFile instances using SourceFileBuilder for both the initial parsing and the final compilation steps, maintaining the proper API usage throughout.

vm/src/stdlib/ast/constant.rs (3)

4-4: LGTM: Import statement correctly added

The import of ruff_source_file::SourceFile is properly added to support the type migration.

101-119: LGTM: Node trait implementations consistently updated

All Node trait implementations for Constant and ConstantLiteral have been systematically updated to:

  • Replace source_code: &SourceCodeOwned parameters with source_file: &SourceFile
  • Update all internal method calls to use the new parameter
  • Maintain identical functionality and logic

The refactoring is complete and consistent across all implementations.

Also applies to: 123-136, 140-167, 171-244

333-398: LGTM: Helper functions consistently updated

All helper functions (number_literal_to_object, string_literal_to_object, bytes_literal_to_object, boolean_literal_to_object, none_literal_to_object, ellipsis_literal_to_object) have been systematically updated with the new source_file: &SourceFile parameter type and all calls are properly updated.

vm/src/stdlib/ast/pattern.rs (2)

2-2: LGTM: Import statement correctly added

The import of ruff_source_file::SourceFile is properly added to support the type migration.

6-48: LGTM: Comprehensive Node trait implementation updates

All Node trait implementations for pattern-related AST types have been systematically updated to:

  • Replace source_code: &SourceCodeOwned parameters with source_file: &SourceFile
  • Update all internal method calls (ast_to_object, ast_from_object, node_add_location, range_from_object) to use the new parameter
  • Maintain identical functionality and control flow

The refactoring covers all pattern matching AST node types including:

  • ruff::MatchCase
  • ruff::Pattern and its variants
  • ruff::PatternMatchValue, ruff::PatternMatchSingleton, etc.
  • Helper structs with placeholder implementations

The migration is complete and consistent throughout the file.

Also applies to: 52-125, 128-157, 161-193, 196-207, 211-243, 247-293, 297-360, 367-378, 385-396, 403-414, 417-444, 448-481, 485-513