code object linetable by youknowone · Pull Request #6150 · RustPython/RustPython

coderabbitai

Walkthrough

Adds CPython 3.11+ linetable and exception table support across the compiler, core, marshal, and VM: codegen collects per-instruction locations and encodes a linetable, core bytecode types gain linetable/exceptiontable fields and marshal persists them, and the VM decodes and exposes these tables via new PyCode APIs.

Changes

Cohort / File(s)	Summary
Core bytecode types & enums `compiler/core/src/bytecode.rs`	Adds public enum `PyCodeLocationInfoKind` with helpers; extends `CodeObject<C>` with `pub linetable: Box<[u8]>` and `pub exceptiontable: Box<[u8]>`; updates `map_bag` and `map_clone_bag` to propagate/clone the new fields.
Compiler IR codegen `compiler/codegen/src/ir.rs`	Collects per-instruction `SourceLocation` entries; implements `generate_linetable` with helpers `write_varint`/`write_signed_varint`, supports Short/OneLine/NoColumns/Long encodings and grouping up to 8 instructions; wires `linetable: Box<[u8]>` into `CodeObject` and initializes `exceptiontable` as an empty boxed slice (TODO: generate real table).
Marshal serialization/deserialization `compiler/core/src/marshal.rs`	`serialize_code` now writes `linetable` and `exceptiontable` as length-prefixed byte slices; `deserialize_code` reads these slices and initializes `CodeObject` with boxed tables.
VM builtins — PyCode exposure & decoding `vm/src/builtins/code.rs`	Adds `LineTableReader`, `PyCodeAddressRange`, imports `PyCodeLocationInfoKind`; extends `ReplaceArgs` and `replace()` to accept/propagate linetable/exceptiontable and related metadata; adds `co_linetable`, `co_exceptiontable`, `co_lines`, `co_positions`; implements decoding/iteration that yields merged (start_offset,end_offset,lineno) ranges and per-instruction position tuples.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Frontend as Compiler Frontend
  participant IR as IR Builder
  participant Codegen as Codegen (ir.rs)
  participant Core as CodeObject (core/bytecode.rs)
  participant Marshal as Marshal
  participant VM as VM PyCode (builtins/code.rs)

  Frontend->>IR: Emit instructions with source locations
  IR->>Codegen: finalize_code(locations, instrs)
  Codegen->>Codegen: generate_linetable(locations) -> Box<[u8]>
  Codegen->>Core: Construct CodeObject{..., linetable, exceptiontable}
  Core->>Marshal: serialize_code includes linetable & exceptiontable
  Marshal->>Core: deserialize_code reads linetable & exceptiontable
  Core->>VM: Load CodeObject into PyCode (contains linetable)
  VM->>VM: LineTableReader parses linetable, yields positions/ranges
  VM-->>Caller: co_linetable()/co_exceptiontable() and co_lines()/co_positions()

sequenceDiagram
  autonumber
  participant PyCode as PyCode.co_lines()
  participant Reader as LineTableReader
  participant Kind as PyCodeLocationInfoKind

  PyCode->>Reader: init with linetable bytes
  loop per encoded entry
    Reader->>Reader: read varint / signed varint
    Reader->>Kind: decode entry type (Short/OneLine/NoColumns/Long/None)
    Kind-->>Reader: provide delta semantics (line/col/no-line)
    Reader-->>PyCode: emit (start_off, end_off, lineno) or position tuple
  end
  PyCode-->>Caller: iterator of merged ranges or positions

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Refactor compile_function #6001 — modifies CodeInfo and finalize_code in compiler/codegen/src/ir.rs, likely overlapping changes to code finalization and linetable handling.

Poem

I nibbled bytes and lines with care,
Packed deltas small in tidy lair.
From IR hops to VM’s view,
Short, Long, OneLine—rules so new.
Exception bed waits for its seed— I hop to stitch the feed. 🐇

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 48.39% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title "code object linetable" succinctly reflects the primary change in the PR—adding linetable support and related fields/APIs to code objects—so it is directly related to the main changeset and clear to a reviewer. It is short, specific, and avoids noisy or vague wording.

✨ Finishing touches

📝 Generate Docstrings

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}