Bytecode parity#7535
Conversation
… genexpr StopIteration wrapper - split_blocks_at_jumps: split blocks at branch points so each has one exit - jump_threading: thread jumps through single-jump blocks (flowgraph.c jump_thread) - Backward conditional jump normalization: invert and create NOT_TAKEN+JUMP block - Follow empty blocks in jump-to-return optimization (next_nonempty_block) - Add PEP 479 StopIteration handler to compile_comprehension for generators
- Add Slice variant to ConstantData and BorrowedConstant - Fold constant slices (x[:3], x[1:4]) into LOAD_CONST(slice(...)) - Marshal serialization/deserialization for Slice type - Box::leak in borrow_obj_constant for PySlice roundtrip
Add Frozenset to ConstantData, BorrowedConstant, and marshal support. Actual frozenset folding (BUILD_SET + CONTAINS_OP → LOAD_CONST frozenset) requires VirtualMachine for element hashing and is deferred.
📝 WalkthroughWalkthroughThis PR extends the compiler's constant handling to support slice and frozenset literals at compile-time, implements PEP 479-style StopIteration exception handling for generators and coroutines, and optimizes the control flow graph with new passes for block splitting, jump threading, and unreachable code elimination. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Sorry, something went wrong.
Prepare infrastructure for exit block duplication optimization. Currently disabled pending stackdepth integration.
0cd225f to
42b24cb
Compare
March 30, 2026 02:43
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (4)
crates/codegen/src/compile.rs (1)
8416-8431: Extract the StopIteration wrapper into a shared helper.This setup/teardown sequence is now effectively duplicated from
Compiler::compile_function_bodyat Lines 3877-3892 and Lines 3924-3939. Any later tweak to handler depth, location suppression, or block layout now has to stay in sync in two places.♻️ Suggested shape
- let stop_iteration_block = if is_gen_scope { - let handler_block = self.new_block(); - emit!( - self, - PseudoInstruction::SetupCleanup { - delta: handler_block - } - ); - self.set_no_location(); - self.push_fblock(FBlockType::StopIteration, handler_block, handler_block)?; - Some(handler_block) - } else { - None - }; + let stop_iteration_block = self.enter_stop_iteration_wrapper(is_gen_scope)?; ... - if let Some(handler_block) = stop_iteration_block { - emit!(self, PseudoInstruction::PopBlock); - self.set_no_location(); - self.pop_fblock(FBlockType::StopIteration); - self.switch_to_block(handler_block); - emit!( - self, - Instruction::CallIntrinsic1 { - func: oparg::IntrinsicFunction1::StopIterationError - } - ); - self.set_no_location(); - emit!(self, Instruction::Reraise { depth: 1u32 }); - self.set_no_location(); - } + self.exit_stop_iteration_wrapper(stop_iteration_block);Also applies to: 8522-8538
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/codegen/src/compile.rs` around lines 8416 - 8431, Extract the duplicated StopIteration setup/teardown into a shared helper (e.g., a method on the compiler like setup_stop_iteration_handler or wrap_with_stopiteration) that encapsulates the logic currently using is_gen_scope, self.new_block(), emitting PseudoInstruction::SetupCleanup { delta: handler_block }, self.set_no_location(), and self.push_fblock(FBlockType::StopIteration, handler_block, handler_block) and returns Option<handler_block>; then replace the duplicated blocks that create stop_iteration_block (the sequences in compile_function_body and the other location around is_gen_scope/is_async) with a call to that helper so both sites share the same behavior and return value.crates/codegen/src/ir.rs (1)
228-229: Consider adding a TODO comment explaining whyduplicate_exits_without_linenois disabled.The comment "pending stackdepth fix" is helpful, but linking to a tracking issue would make it easier to follow up.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/codegen/src/ir.rs` around lines 228 - 229, Update the existing TODO comment above the disabled call to duplicate_exits_without_lineno(&mut self.blocks) to include a reference to the tracking issue or bug ID and a short timestamp/owner tag (e.g., "pending stackdepth fix - see ISSUE-1234 `@2026-03-30`") so future readers can find the follow-up; leave the call commented out but expand the comment to mention the exact reason ("stackdepth calculation bug") and the issue link/number and optionally who filed it.crates/vm/src/builtins/code.rs (1)
319-326: Theunimplemented!panic is appropriately documented.The comment clearly explains that
PyObjBaglacksVirtualMachineaccess needed for frozenset hashing, and thatPyMarshalBaghandles this case. Consider adding a link to wherePyMarshalBagimplements this, or logging a warning if this path is ever hit unexpectedly.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/vm/src/builtins/code.rs` around lines 319 - 326, In the BorrowedConstant::Frozenset arm inside PyObjBag::make_constant, replace the bare unimplemented!() with a logged warning that clearly states this path should be handled by PyMarshalBag (reference PyMarshalBag's frozenset handling in the message) and include a link or code reference to where PyMarshalBag implements it; after logging, either return an appropriate Err or keep the unimplemented panic if you want to preserve current behavior, but ensure the warning is emitted first so unexpected hits are observable.crates/compiler-core/src/bytecode.rs (1)
934-935: Note:Frozensethash implementation doesn't account for set semantics.The current hash implementation hashes elements in order, but frozensets are unordered. If two
Frozensetconstants have the same elements in different orders, they would have different hashes but should compare equal. However, since this is used for constant pool deduplication and constants are typically created in a deterministic order, this may be acceptable.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/compiler-core/src/bytecode.rs` around lines 934 - 935, The Frozenset arm currently calls elements.hash(state) which is order-dependent; change it to an order-independent hashing strategy so frozensets with the same elements in different orders produce identical hashes. In the Frozenset match arm (the Frozenset { elements } branch inside the Hash impl for constants), compute each element's hash into a temporary value, sort those element-hash values (or otherwise combine them with a commutative operation like XOR/add on u64 hashes), then feed the sorted/combined result into the provided state so hashing is deterministic and order-independent.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@crates/codegen/src/compile.rs`:
- Around line 7577-7581: try_fold_constant_slice currently only recognizes
numbers, booleans and None and limits integer folding to i64, which is narrower
than ExprExt::is_constant_slice; update try_fold_constant_slice to accept the
same constant kinds that ExprExt::is_constant_slice treats constant (include
string/bytes/ellipsis bounds and arbitrary-size integers, not just i64) so that
patterns like obj["a":"z"] and large integer literals are folded to
LOAD_CONST(slice) and hit the BINARY_SLICE fast path; make the same expansion in
the adjacent constant-folding block that has the i64 restriction so both places
have parity with ExprExt::is_constant_slice.
In `@crates/compiler-core/src/bytecode.rs`:
- Around line 854-887: The From<ConstantData> for StackValue implementation
currently uses a wildcard arm and will panic for the new ConstantData::Slice and
ConstantData::Frozenset variants; update the match in From<ConstantData> for
StackValue to explicitly handle ConstantData::Slice (map it to an appropriate
StackValue variant representing a slice object, constructing start/stop/step
from the elements array) and ConstantData::Frozenset (convert elements
Vec<ConstantData> into the corresponding StackValue frozenset representation),
ensuring you pattern-match on ConstantData::Slice and ConstantData::Frozenset
instead of relying on the unimplemented wildcard arm.
In `@crates/compiler-core/src/marshal.rs`:
- Around line 471-478: The make_frozenset implementation collects elements into
a Vec and passes them straight to make_constant, which preserves duplicates;
update make_frozenset to deduplicate elements before constructing the
BorrowedConstant::Frozenset so its behavior matches CPython. Concretely, in
make_frozenset use a set (e.g., HashSet or BTreeSet) to track seen values and
build a new Vec of unique elements (preserving a deterministic order such as
first-seen or sorted if needed), then call
self.make_constant::<Bag::Constant>(BorrowedConstant::Frozenset { elements:
&unique_elements }) instead of passing the original elements Vec. Ensure you
reference the existing function make_frozenset and the constant constructor
BorrowedConstant::Frozenset when applying the change.
---
Nitpick comments:
In `@crates/codegen/src/compile.rs`:
- Around line 8416-8431: Extract the duplicated StopIteration setup/teardown
into a shared helper (e.g., a method on the compiler like
setup_stop_iteration_handler or wrap_with_stopiteration) that encapsulates the
logic currently using is_gen_scope, self.new_block(), emitting
PseudoInstruction::SetupCleanup { delta: handler_block },
self.set_no_location(), and self.push_fblock(FBlockType::StopIteration,
handler_block, handler_block) and returns Option<handler_block>; then replace
the duplicated blocks that create stop_iteration_block (the sequences in
compile_function_body and the other location around is_gen_scope/is_async) with
a call to that helper so both sites share the same behavior and return value.
In `@crates/codegen/src/ir.rs`:
- Around line 228-229: Update the existing TODO comment above the disabled call
to duplicate_exits_without_lineno(&mut self.blocks) to include a reference to
the tracking issue or bug ID and a short timestamp/owner tag (e.g., "pending
stackdepth fix - see ISSUE-1234 `@2026-03-30`") so future readers can find the
follow-up; leave the call commented out but expand the comment to mention the
exact reason ("stackdepth calculation bug") and the issue link/number and
optionally who filed it.
In `@crates/compiler-core/src/bytecode.rs`:
- Around line 934-935: The Frozenset arm currently calls elements.hash(state)
which is order-dependent; change it to an order-independent hashing strategy so
frozensets with the same elements in different orders produce identical hashes.
In the Frozenset match arm (the Frozenset { elements } branch inside the Hash
impl for constants), compute each element's hash into a temporary value, sort
those element-hash values (or otherwise combine them with a commutative
operation like XOR/add on u64 hashes), then feed the sorted/combined result into
the provided state so hashing is deterministic and order-independent.
In `@crates/vm/src/builtins/code.rs`:
- Around line 319-326: In the BorrowedConstant::Frozenset arm inside
PyObjBag::make_constant, replace the bare unimplemented!() with a logged warning
that clearly states this path should be handled by PyMarshalBag (reference
PyMarshalBag's frozenset handling in the message) and include a link or code
reference to where PyMarshalBag implements it; after logging, either return an
appropriate Err or keep the unimplemented panic if you want to preserve current
behavior, but ensure the warning is emitted first so unexpected hits are
observable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: 54e2036a-fe74-4e78-9275-4b95f948e8d1
⛔ Files ignored due to path filters (1)
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__nested_double_async_with.snapis excluded by!**/*.snap
📒 Files selected for processing (5)
crates/codegen/src/compile.rscrates/codegen/src/ir.rscrates/compiler-core/src/bytecode.rscrates/compiler-core/src/marshal.rscrates/vm/src/builtins/code.rs
Sorry, something went wrong.
3706c53
into
RustPython:main
Mar 30, 2026
Summary by CodeRabbit
New Features
Bug Fixes
Performance