◐ Shell
reader mode source ↗
Skip to content

Update opcode, dis from 3.14.2#6869

Merged
youknowone merged 6 commits into
RustPython:mainfrom
youknowone:opcode
Jan 26, 2026
Merged

Update opcode, dis from 3.14.2#6869
youknowone merged 6 commits into
RustPython:mainfrom
youknowone:opcode

Conversation

@youknowone

@youknowone youknowone commented Jan 25, 2026

Copy link
Copy Markdown
Member

Summary by CodeRabbit

  • New Features

    • Added a new public API to access bytecode data.
  • Refactor

    • Improved code-generation pipeline with earlier optimizations and cleaner bytecode processing.
    • Standardized resume instruction emission across scopes for more consistent behavior.
    • Enhanced source-location/linetable tracking to improve debugging and error reporting.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai

coderabbitai Bot commented Jan 25, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR centralizes Resume emission into a new emit_resume_for_scope helper on the compiler, adds per-instruction line-table scaffolding (LineTableLocation and lineno_override) and moves several codegen optimizations (tuple-constant folding, small-int conversion, unused-const/NOP removal) earlier in finalization. It also adds a _co_code_adaptive getter on PyCode.

Changes

Cohort / File(s) Change Summary
Resume emission centralization
crates/codegen/src/compile.rs
Added emit_resume_for_scope(scope_type, lineno) and refactored multiple sites to call it; standardized Resume instruction payload (location, end_location, except_handler, lineno_override) and removed inlined SourceLocation creation in callers.
Linetable & IR pipeline
crates/codegen/src/ir.rs
Added LineTableLocation and pub lineno_override: Option<i32> on InstructionInfo; updated finalize pipeline to perform tuple constant folding, LOAD_CONSTLOAD_SMALL_INT conversion for module-level ints, unused-const removal, earlier NOP stripping, and to accumulate per-instruction linetable locations for final linetable generation.
VM code object API
crates/vm/src/builtins/code.rs
Added public getter pub fn _co_code_adaptive(&self, vm: &VirtualMachine) -> PyBytesRef that delegates to co_code.

Sequence Diagram(s)

(omitted)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • Instruction::Resume #5944 — Related changes inserting Resume emission at yields and function starts; overlaps resume insertion behavior.
  • compiler enter_scope #5950 — Changes to enter_scope and RESUME emission; closely related to centralizing resume logic.
  • code object linetable #6150 — Modifies linetable/InstructionInfo handling and codegen pipeline; intersects with lineno_override and linetable generation.

Suggested reviewers

  • ShaharNaveh

Poem

🐰 I hopped through code with nibbling paws,

Resumes now gather under one small clause,
Lines draw maps where bytecodes dance,
Small ints skipped in a swift advance,
A carrot-cheer for tidy chance! 🥕

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 69.23% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: updating opcode and dis modules to align with Python 3.14.2, which is clearly reflected in the commits involving Resume instruction changes, linetable generation, and code object adaptations.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@youknowone youknowone force-pushed the opcode branch 2 times, most recently from 12b9e0e to cd978d9 Compare January 25, 2026 11:45
@youknowone youknowone marked this pull request as ready for review January 25, 2026 12:16

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Actionable comments posted: 3

🤖 Fix all issues with AI agents
In `@crates/codegen/src/compile.rs`:
- Around line 1145-1187: emit_resume_for_scope currently resets
self.current_source_range to TextRange::default() before emitting RESUME, so
non-module scopes lose the actual definition line; update the call site (where
current_source_range is computed) to pass the real definition line into
emit_resume_for_scope (e.g., add a lineno: Option<u32> parameter) and inside
emit_resume_for_scope set lineno_override = Some(lineno) for non-module scopes
(or, alternatively, restore/populate self.current_source_range from that
definition range before emitting), ensuring the RESUME entry uses the correct
lineno; modify the emit_resume_for_scope signature and its callers accordingly
(refer to emit_resume_for_scope, current_source_range, and the RESUME
instruction emission).

In `@crates/codegen/src/ir.rs`:
- Around line 226-229: When handling a module-scope RESUME with lineno_override
== Some(0), you must also clear end_line and any column information so the
resulting LineTableLocation represents "no line" (line 0, no columns) per
CPython 3.13+ semantics. Update the code that constructs or pushes into
linetable_locations (and any related use of instructions/locations) to, when
lineno_override == Some(0), set start_line and end_line to 0 and clear/reset
start_col/end_col (or their equivalent fields in LineTableLocation) so no column
data is emitted for that entry.

In `@crates/stdlib/src/_opcode.rs`:
- Around line 328-343: The get_common_constants function currently swallows all
errors when calling vm.builtins.get_attr("all"/"any") via unwrap_or_else(|_|
vm.ctx.none()), which masks allocation/internal errors and diverges from frame
opcode behavior; change the signature of get_common_constants to return
PyResult<Vec<PyObjectRef>> (instead of Vec<...>) and propagate get_attr errors
with the ? operator so failures bubble up, or if the intent is to only treat
missing builtins as None explicitly, replace unwrap_or_else with .ok() and map
AttributeError to vm.ctx.none while still propagating other errors; update call
sites to handle the new PyResult as needed and reference get_common_constants,
get_attr, vm.builtins, vm.ctx.none, and VirtualMachine when making the change.
🧹 Nitpick comments (1)
crates/codegen/src/ir.rs (1)

171-186: Optimization helpers and finalize_code integration look good.

The fold/convert/cleanup helpers keep the optimization pipeline readable. Operational note: after bytecode instruction changes, run a full clean build and remove any .pyc artifacts to avoid stale caches. Based on learnings, please run rm -r target/debug/build/rustpython-* and delete any .pyc files.

Also applies to: 444-521, 587-617, 619-679, 681-688

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@crates/codegen/src/compile.rs`:
- Around line 1152-1188: The call to enter_scope is passing a byte-offset
(expr.range().start().to_u32()) which enter_scope treats as a line number and
corrupts the line table; locate the call site where enter_scope(...) is invoked
with expr.range().start().to_u32() (around the code that handles expression
scopes) and replace that argument with the actual source line number:
self.get_source_line_number().get().to_u32(), so enter_scope and downstream
SourceLocation creation receive a proper line number consistent with other
callers.
♻️ Duplicate comments (1)
crates/codegen/src/ir.rs (1)

335-344: Override end_line and columns when lineno_override is set to 0.

When lineno_override = Some(0) is set for module-scope RESUME, only line is overridden to 0 while end_line, col, and end_col retain the actual source location. Per CPython 3.13+ semantics, line 0 represents "no line" (artificial instructions) and should have no meaningful column information.

🔧 Suggested fix
-                    let lt_loc = LineTableLocation {
-                        line: info
-                            .lineno_override
-                            .unwrap_or_else(|| info.location.line.get() as i32),
-                        end_line: info.end_location.line.get() as i32,
-                        col: info.location.character_offset.to_zero_indexed() as i32,
-                        end_col: info.end_location.character_offset.to_zero_indexed() as i32,
-                    };
+                    let line_override = info.lineno_override;
+                    let line = line_override.unwrap_or_else(|| info.location.line.get() as i32);
+                    let (end_line, col, end_col) = if line_override == Some(0) {
+                        // Line 0 = no location info per CPython semantics
+                        (0, 0, 0)
+                    } else {
+                        (
+                            info.end_location.line.get() as i32,
+                            info.location.character_offset.to_zero_indexed() as i32,
+                            info.end_location.character_offset.to_zero_indexed() as i32,
+                        )
+                    };
+                    let lt_loc = LineTableLocation { line, end_line, col, end_col };

@github-actions

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[+] lib: cpython/Lib/dis.py
[+] test: cpython/Lib/test/test_dis.py

dependencies:

  • dis

dependent tests: (49 tests)

  • dis: test__opcode test_ast test_code test_compile test_compiler_assemble test_dis test_dtrace test_fstring test_inspect test_opcache test_patma test_peepholer test_positional_only_arg
    • inspect: test_abc test_argparse test_asyncgen test_builtin test_collections test_decimal test_enum test_functools test_generators test_grammar test_ntpath test_operator test_posixpath test_signal test_traceback test_types test_typing test_unittest test_yield_from test_zipimport
      • asyncio: test_contextlib_async test_logging test_os test_sys_settrace test_unittest
      • bdb: test_bdb
      • dataclasses: test__colorize test_genericalias test_pprint test_regrtest test_zoneinfo
      • importlib.metadata: test_importlib
      • rlcompleter: test_rlcompleter
      • trace: test_trace
      • xmlrpc.server: test_docxmlrpc test_xmlrpc

[+] lib: cpython/Lib/opcode.py
[+] lib: cpython/Lib/_opcode_metadata.py
[+] test: cpython/Lib/test/test__opcode.py
[+] test: cpython/Lib/test/test_opcodes.py

dependencies:

  • opcode (native: _opcode, builtins)
    • _opcode_metadata
    • _opcode_metadata

dependent tests: (34 tests)

  • opcode: test__opcode test_code test_dis test_peepholer
    • dis: test_ast test_compile test_compiler_assemble test_dtrace test_fstring test_inspect test_opcache test_patma test_positional_only_arg
      • inspect: test_abc test_argparse test_asyncgen test_builtin test_collections test_decimal test_enum test_functools test_generators test_grammar test_ntpath test_operator test_posixpath test_signal test_traceback test_types test_typing test_unittest test_yield_from test_zipimport
      • trace: test_trace

[+] test: cpython/Lib/test/test_compile.py

dependencies:

dependent tests: (no tests depend on compile)

[+] test: cpython/Lib/test/test_peepholer.py

dependencies:

dependent tests: (no tests depend on peepholer)

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

Hide details View details @youknowone youknowone merged commit 93b26bf into RustPython:main Jan 26, 2026
14 checks passed
@youknowone youknowone deleted the opcode branch January 26, 2026 02:39
This was referenced Feb 22, 2026
This was referenced Mar 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant