◐ Shell
reader mode source ↗
Skip to content

Await fsm checkpoint#6765

Draft
yusufyian wants to merge 47 commits into
RustPython:mainfrom
yusufyian:await_fsm_checkpoint
Draft

Await fsm checkpoint#6765
yusufyian wants to merge 47 commits into
RustPython:mainfrom
yusufyian:await_fsm_checkpoint

Conversation

@yusufyian

@yusufyian yusufyian commented Jan 18, 2026

Copy link
Copy Markdown

Summary by CodeRabbit

  • New Features

    • Checkpoint and resume functionality to save/restore VM execution state
    • FSM-based async function continuations with decorator-driven transformations
    • Filesystem-backed host API with state persistence, gas accounting, and event logging
    • Deterministic execution mode with configurable stdlib whitelisting
    • Binary renamed from rustpython to pvm
  • Examples

    • Smart contract demos: DEX with AMM mechanics, staking rewards, escrow marketplace, batch payroll
    • Checkpoint/resume workflow demonstrations

✏️ Tip: You can customize this high-level summary in your review settings.

shawhanken and others added 30 commits December 18, 2025 17:13
- Implemented `checkpoint_stack` and `restore_stack` methods in the Frame struct for managing stack checkpoints.
- Added `run_script_resume` method in VirtualMachine to resume execution from a checkpoint.
- Introduced `resume_path` option in Settings to specify the checkpoint file path.
- Updated command-line arguments to support `--resume` for checkpoint execution.
- Registered the new `rustpython_checkpoint` module in the standard library.
- Introduced `maybe_checkpoint_request` function to handle checkpoint requests during execution.
- Updated `checkpoint` function to set a checkpoint request with the expected last instruction index.
- Enhanced `save_checkpoint` to validate the stack state and added a new `save_checkpoint_from_exec` function for saving checkpoints from execution context.
- Modified global state to include a mutex for checkpoint requests.
- Updated demo and README to reflect changes in checkpoint handling and serialization requirements.
- Added multiple print statements to display the `imhere` variable at various stages of execution.
- Updated the `imhere` variable to include new messages for better tracking of execution flow.
- Changed the demo user from "alice" to "Tony" for better context.
- Added a section in the README for running automated tests to verify checkpoint resume functionality.
- Added demo.dot and demo.png to .gitignore to prevent tracking of demo-related files.
- Added references for Cowboy PVM Task Plan to .gitignore to prevent tracking.
- Updated import statement in demo.py to include type ignore for rustpython_checkpoint, ensuring compatibility with type checkers.
…r improved clarity and context. Changed user references in demo script and enhanced comments for better understanding of checkpoint phases.
…r improved clarity and context. Changed user references in demo script and enhanced comments for better understanding of checkpoint phases.
…tpython' to 'pvm'

- Modified README instructions and examples to use 'pvm' for running the demo and tests.
- Updated the test script to resolve the binary path to 'pvm' instead of 'rustpython'.
- Enhanced clarity in README regarding the prototype's limitations.
…process

- Updated the demo script to enhance the presentation of phases in the checkpoint/resume process.
- Reorganized print statements to provide clearer context and summaries of the runs and costs.
- Improved comments to better explain the purpose of each phase and the checkpointing mechanism.
- Ensured that the script maintains a consistent format for logging and output.
- Modified demo.py to simulate a trading day pipeline, including order loading, risk checks, and trade settlement.
- Enhanced print statements for clarity on order processing, risk flags, and final report generation.
- Updated README to describe the new financial trading scenario and its phases, ensuring users understand the demo's functionality.
…README for clarity

- Expanded demo.py to include new features for simulating trading scenarios, such as enhanced order processing and risk management.
- Improved print statements for better visibility into the trading workflow and outcomes.
- Revised README to provide clearer instructions and context for the updated trading scenario functionality.
- Added `pvm-host` crate for Host API definitions and error types.
- Introduced `pvm-runtime` crate for PVM runtime encapsulation, including new features and modules.
- Updated Cargo.lock to reflect the addition of `pvm-host` and `pvm-runtime` dependencies.
- Modified `.cspell.json` and `.cspell.dict` files to enhance spell checking capabilities.
- Updated `Cargo.toml` and `Cargo.lock` to reflect dependency changes.
- Enhanced `whats_left.py` for better argument handling.
- Refined CI workflows in `.github` for improved automation.
- Added new test cases and updated existing ones across multiple modules to ensure comprehensive coverage.
- Cleaned up and organized various source files for better readability and maintainability.
- Added a new reference file to `.gitignore` for better source management.
- Updated `frame.rs` and `checkpoint.rs` to allow dead code, improving future development flexibility.
- Refined condition checks in `checkpoint.rs` and `frame.rs` to use `PopTop` for better instruction handling.
- Improved warning logging in `core.rs` to check for VM presence before logging errors.
- Adjusted `compile.rs` to access configuration settings more clearly.
- Made `checkpoint_request` field in `PyGlobalState` private for encapsulation.
- Introduced a new utility function in `thread.rs` to check VM stack status.
- Cleaned up `interpreter.rs` by removing unused imports for better readability.
- Updated `lib.rs` to access resume path settings through the new configuration structure.
- Added a new reference file to `.gitignore` for improved source management.
- Introduced a helper function in `demo_en.py` to format section titles after resuming from checkpoints, enhancing output clarity.
- Updated the binary snapshot file `demo_en.rpsnap` to reflect changes in the demo script.
…entries

- Added a wildcard entry to `.gitignore` to ignore all reference files for better source management.
- Removed specific entries from `.gitignore` to streamline the ignore list.
…design

- Deleted `PVM_CHAIN_INTEGRATION_CN.md`, `PVM_Continuation_Design_CN.md`, and `PVM_Feasibility_and_Roadmap_CN.md` as they are no longer needed for the current project direction.
- This cleanup helps streamline the repository and focus on active development areas.
- Deleted `demo_en.py`, `demo_en.rpsnap`, and associated README files as they are no longer relevant to the current project direction.
- This cleanup streamlines the repository and focuses on active development areas, removing outdated examples and documentation.
- Deleted `demo.rpsnap` as it is no longer relevant to the current project direction.
- This cleanup continues to streamline the repository by removing outdated demo artifacts.
- Deleted `test_checkpoint.py` as it is no longer relevant to the current project direction.
- This cleanup continues to streamline the repository by removing outdated test artifacts.
- Updated comments in `state_store.py` to provide clear English descriptions of the functions' purposes, enhancing code readability and maintainability.
- Added a new entry to `.gitignore` for the `demo.rpsnap` file to improve source management.
- Refactored `checkpoint.rs` to streamline checkpoint saving and loading processes, including the introduction of `save_checkpoint_bytes_from_exec` for better data handling.
- Updated `save_checkpoint_from_exec` to accept a `code` parameter, enhancing flexibility in checkpoint management.
- Marked several functions as `#[allow(dead_code)]` to facilitate future development without immediate usage requirements.
- Added `PositionEncoding` to the `source_location` method in `SourceCode` for improved text encoding support.
- Updated the public use statement to include `PositionEncoding` from `ruff_source_file`.
- Updated `save_checkpoint_from_exec` to ensure data is written correctly by passing a reference to `data`.
- Enhanced `SnapshotWriter` and `SnapshotReader` to support additional object types and improve serialization/deserialization processes.
- Introduced caching mechanisms for type attributes and instance data to optimize performance during snapshot operations.
- Improved error handling and added cycle detection during object restoration to prevent infinite loops.
- Refactored global resolution logic in `SnapshotReader` for better handling of function globals.
- Cleaned up demo script by removing unnecessary imports and comments for clarity.
- Changed the version string format to include "PVM 0.0.2" for clarity in version reporting.
- Maintained existing formatting conventions to ensure consistency with previous outputs.
- Introduced a new section in `Cargo.toml` for PVM versioning, setting the current version to "0.0.2".
- Updated `build.rs` to read the PVM version from the environment or `Cargo.toml`, enhancing build configuration.
- Modified `version.rs` to include the PVM version in the output format for better clarity in version reporting.
- Added new methods `get_stack` and `push_stack_value` to the `Frame` struct for improved stack handling during checkpoints.
- Refactored checkpoint saving functions to support multiple frames, allowing for more complex execution states to be captured.
- Updated `save_checkpoint_with_lasti_and_stack` to handle both the instruction pointer and stack state for the innermost frame.
- Enhanced serialization of checkpoint data to include frame states and their respective stacks, improving the robustness of the checkpointing mechanism.
- Introduced a new demo script to showcase the updated checkpoint and resume functionality in a complex actor model scenario.
- Added a new entry to `.gitignore` for `actor_complex_demo.rpsnap` to improve source management.
- Introduced methods `get_stack` and `push_stack_value` in the `Frame` struct to support multi-frame checkpointing.
- Refactored checkpoint saving functions to handle both instruction pointer and stack state for improved robustness.
- Updated demo script `actor_complex_demo.py` to showcase the enhanced checkpoint and resume functionality in a complex actor model scenario.
- Introduced new `ObjTag` variants for `Enumerate`, `Zip`, `Map`, and `Filter` to enhance object type recognition during snapshot operations.
- Updated `ObjectPayload` to include corresponding payload structures for these new object types.
- Implemented serialization and deserialization logic for `Enumerate`, `Zip`, `Map`, and `Filter` to ensure proper restoration of these objects from snapshots.
- Enhanced `classify_obj` function to identify these new iterator types based on their class names.
- Improved error handling and validation during the snapshot process for these new object types, ensuring robustness in state management.
- Made `Block`, `BlockType`, and `UnwindReason` structs public for better accessibility.
- Introduced methods `push_block` and `get_blocks` in the `Frame` struct to manage block stack during checkpoints.
- Updated checkpoint saving functions to include block states and prepared locals, improving the robustness of the checkpointing mechanism.
- Enhanced serialization and deserialization processes to handle block states, ensuring accurate restoration of control flow during execution.
- Refactored demo script to reflect changes in checkpoint handling and improve clarity.
- Introduced new ObjTag variants for ListIterator, RangeIterator, and Range to enhance object type recognition during snapshot operations.
- Updated ObjectPayload to include corresponding payload structures for ListIterator, RangeIterator, and Range.
- Implemented serialization and deserialization logic for ListIterator and RangeIterator, ensuring proper restoration of these objects from snapshots.
- Enhanced classify_obj function to identify new iterator types based on their class names.
- Updated demo script to reflect changes in loop handling and checkpointing, improving clarity and functionality.
- Added `comprehensive_demo.rpsnap` to the `.gitignore` file to improve source management and prevent unnecessary tracking of demo snapshots.
- Introduced `ExecutionOptions` struct to encapsulate execution parameters for better configurability.
- Enhanced error handling in the `execute_tx` and `run_source` functions to map exceptions to `HostError` types.
- Updated the `host_error` function to include error code and name attributes for improved debugging.
- Added new methods for `ExecutionOptions` to facilitate setting parameters fluently.
- Refactored demo scripts to utilize the new execution options, improving clarity and functionality.
- Introduced `DeterminismOptions` to the `ExecutionOptions` struct for improved execution control.
- Added new error types: `DeterministicValidationError`, `NonDeterministicError`, and `OutOfGasError` to enhance error reporting.
- Updated `execute_tx_with_options` to handle determinism settings and associated error handling.
- Enhanced demo scripts to showcase determinism features and provide clearer usage instructions.
- Refactored exception mapping to include determinism-related errors for better debugging and user feedback.
- Added `trace_imports`, `trace_allow_all`, and `trace_path` options to `DeterminismOptions` for improved import tracking.
- Implemented import tracing functionality in the PVM runtime, allowing for detailed logging of module imports.
- Updated `Cargo.toml` to include a new dependency on `pvm-runtime` and incremented the PVM version to "0.1.3".
- Enhanced demo scripts to support new tracing features and provide clearer usage instructions for import tracing.
- Refactored the `determinism_check.py` script to include options for generating and printing suggested import whitelists.
- Included three new demo scripts: `escrow_marketplace_demo.py`, `batch_payroll_demo.py`, and `staking_rewards_demo.py` to showcase various business scenarios.
- Updated the README to provide usage instructions for running the new demos with determinism checks.
- Enhanced clarity in the documentation regarding input options and expected outputs for the demos.
…e at the end of the file for better readability and adherence to coding standards.
- Added support for continuation modes (FSM and checkpoint) in the PVM runtime, allowing for more flexible execution control.
- Introduced new methods for sending messages, scheduling timers, and canceling timers in the `FsHost` struct.
- Updated `HostContext` to include additional fields for actor address, message ID, and nonce.
- Enhanced the `ExecutionOptions` struct to accommodate continuation options and updated related functions for better configurability.
- Refactored demo scripts to showcase new continuation features and provide clearer usage instructions.
- Improved error handling and state management during checkpoint operations, ensuring robust execution flow.
@coderabbitai

coderabbitai Bot commented Jan 18, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR introduces a comprehensive portable VM (PVM) execution environment with support for deterministic Python execution, continuation-based async workflows, and filesystem-backed state management. It adds new crates (pvm-alto, pvm-host, pvm-runtime) providing a host abstraction layer, configurable execution options, checkpoint/resume capabilities, and multiple example contracts demonstrating smart-contract-like operations.

Changes

Cohort / File(s) Summary
Host Interface Abstraction
crates/pvm-host/Cargo.toml, crates/pvm-host/src/lib.rs
New crate defining HostApi trait with state management (get/set/delete), event emission, gas accounting, randomness, messaging, and timer operations; includes HostContext, HostError types, and HostResult type alias.
Filesystem Host Implementation
crates/pvm-alto/Cargo.toml, crates/pvm-alto/src/lib.rs
New crate implementing FsHost (filesystem-backed HostApi), state directory operations, event logging, gas tracking, and helper FsTxConfig and execute_tx_fs function for transaction execution.
PVM Runtime Engine
crates/pvm-runtime/Cargo.toml, crates/pvm-runtime/src/lib.rs, crates/pvm-runtime/src/continuation.rs, crates/pvm-runtime/src/determinism.rs, crates/pvm-runtime/src/guard.rs, crates/pvm-runtime/src/host.rs, crates/pvm-runtime/src/module.rs
New crate providing Python execution environment with ExecutionOptions, determinism controls, continuation modes, host integration via thread-local HostGuard, and pvm_host_module exposing host operations to Python.
Codegen FSM Transformation
crates/codegen/Cargo.toml, crates/codegen/src/compile.rs, crates/codegen/src/lib.rs, crates/codegen/src/pvm_fsm.rs
Adds FSM continuation transformation in pvm_fsm.rs (503 lines) to convert decorated async functions into paired initial/resume functions; moves ruff_python_parser to main dependencies and adds pvm_fsm flag to CompileOpts.
VM Checkpoint/Resume Core
crates/vm/src/vm/checkpoint.rs, crates/vm/src/vm/mod.rs, crates/vm/src/vm/compile.rs, crates/vm/src/frame.rs
New checkpoint module (477 lines) for saving/restoring multi-frame VM state with various serialization options; adds CheckpointRequest/CheckpointTarget types, run_script_resume methods, checkpoint field accessibility on Frame, and state persistence via snapshot utilities.
VM Configuration & Settings
crates/vm/src/vm/setting.rs, crates/vm/src/vm/thread.rs, crates/vm/build.rs, crates/vm/src/version.rs
Adds ContinuationMode enum, resume_path, continuation_mode, and checkpoint_exit fields to Settings; introduces PVM_VERSION build-time detection; adds has_vm() thread-local checker; updates version string to include PVM version.
VM Integration Points
crates/vm/src/stdlib/mod.rs, crates/vm/src/stdlib/rustpython_checkpoint.rs, crates/vm/src/object/core.rs
Registers rustpython_checkpoint module with checkpoint/checkpoint_bytes functions for Python; conditionalizes del warning on VM thread presence.
Compiler Support
crates/compiler-source/src/lib.rs, examples/dis.rs
Re-exports PositionEncoding; adds CompileOpts.pvm_fsm initialization in example.
JSON Module Enhancement
crates/stdlib/src/json.rs
Makes JsonScanner constructor defaults optional (strict, hooks, parse_constant); adds getnewargs method.
Project Configuration
.gitignore, Cargo.toml, src/lib.rs, src/settings.rs
Updates .gitignore with demo artifacts and state files; adds pvm-alto/host/runtime dev-dependencies; renames binary to "pvm"; adds [package.metadata.pvm] version; adds --resume and --continuation CLI options.
Example Contracts & Demos
examples/pvm_actor_transfer_demo/*, examples/pvm_dex_demo/*, examples/pvm_runtime_chain_demo/*, examples/breakpoint_resume_demo/*, examples/ast_visualize/*
Comprehensive suite of example applications demonstrating actor transfer contracts, DEX operations, staking/payroll systems, checkpoint/resume workflows, and AST visualization utilities.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~70 minutes

Possibly related PRs

  • pythonrun #6643: Refactors VM execution API methods (run_script/run_string relocation) that are enhanced by this PR's resume functionality additions.
  • Fix traceback #6569: Modifies CompileOpts struct with new boolean flags; this PR similarly extends CompileOpts.pvm_fsm in parallel with continuation mode handling.
  • Introduce PyConfig to implement Paths correctly #6461: Modifies VM settings/compile configuration surface; this PR extends Settings with continuation_mode, resume_path, and checkpoint_exit fields.

Suggested reviewers

  • youknowone
  • ShaharNaveh
  • fanninpm

🐰 A Portable VM Takes Flight!

From host abstractions to checkpoints we save,
FSM transformations make async waves behave,
State stored and resumed through filesystem's care,
Determinism assured—a smart contract layer fair! 🚀

🚥 Pre-merge checks | ✅ 1 | ❌ 2
❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.97% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'Await fsm checkpoint' is vague and does not clearly summarize the main changes. The PR introduces a comprehensive PVM runtime system with FSM continuations, checkpointing, host APIs, and extensive examples, but the title uses non-descriptive acronyms without context. Revise the title to more clearly describe the primary objective, such as 'Add PVM runtime with FSM continuations and checkpointing support' or similar, to better convey the scope and intent of these substantial changes.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Actionable comments posted: 20

🤖 Fix all issues with AI agents
In @.gitignore:
- Around line 25-35: Remove the duplicate "Cargo.lock" entry and delete the
self-ignore ".gitignore" entry from the .gitignore file: remove the redundant
"Cargo.lock" line and the line that lists ".gitignore" so the gitignore file is
not ignored itself and the lockfile is only listed once.

In `@Cargo.toml`:
- Around line 69-71: The Cargo.toml binary was renamed to "pvm" (the [[bin]]
entry with name = "pvm" and path = "src/main.rs"), which breaks user docs and
packaging; update all downstream references to use the new binary name or
provide a compatibility shim: replace occurrences of "rustpython" with "pvm" in
README.md, wapm.toml, and scripts/redox/run-redoxer.sh, and/or add a
compatibility alias (e.g., create a wrapper script or symlink named "rustpython"
that invokes "pvm") so existing install/usage examples and packaging continue to
work.

In `@crates/pvm-runtime/src/determinism.rs`:
- Around line 23-99: In default_whitelist() remove the "_thread" entry from the
returned Vec<String> and re-run/validate that no pvm_sdk modules (e.g., pvm_sdk,
pvm_sdk.runtime, pvm_sdk.runner, pvm_sdk.actor, pvm_sdk.verify, pvm_sdk.types)
or other allowed modules depend on low-level threading; if any depend, either
add a documented justification for keeping "_thread" in default_whitelist() or
replace the dependency with a PVM-controlled stub (implementing a controlled API
in pvm_sdk like pvm_sdk.pvm_thread that disables real OS thread spawning) before
keeping it.

In `@crates/pvm-runtime/src/host.rs`:
- Around line 14-35: The HostGuard.install currently overwrites TLS (HOST,
RUNTIME_CONFIG) and Drop unconditionally clears them; change HostGuard to save
the previous TLS contents when install(host, runtime_config) runs (read
HOST.with(|c| c.get()) and RUNTIME_CONFIG.with(|c| c.get()) before setting the
new values), store those previous values as fields on HostGuard (e.g. prev_host:
Option<HostPtr>, prev_runtime: Option<RuntimeConfig>), and modify Drop for
HostGuard to restore those saved values into the TLS cells instead of setting
None (use HOST.with(|c| c.set(self.prev_host)) and similarly for
RUNTIME_CONFIG). Ensure types match (HostPtr and RuntimeConfig) and keep the
existing pointer transmute logic in install.

In `@crates/vm/build.rs`:
- Around line 86-102: The version parsing in build.rs currently returns the raw
value which can include inline comments or single quotes; update the block in
the loop that handles key == "version" to sanitize the value by first removing
any inline comments (e.g., split on '#' and/or '//' and take the left side),
then trim whitespace and strip both double and single quote characters
(trim_matches('"').trim_matches('\'') or equivalent) before returning; reference
the in_pvm_section handling and the key == "version" branch so you only change
the value sanitization logic there.

In `@crates/vm/src/frame.rs`:
- Around line 500-512: The current checkpoint code silently drops locals when
self.fastlocals.try_lock() fails; change this to acquire the lock in a
non-silent way: replace try_lock() with a blocking lock (self.fastlocals.lock())
inside the current_locals block and handle the PoisonError (or lock failure)
explicitly—either propagate an Err from the surrounding checkpoint/Frame method
or log a clear warning before returning/continuing. Ensure you update the
current_locals construction that uses locals_dict and the loop over
self.code.code.varnames so it only proceeds after a successful lock and include
the error handling/logging path for the lock acquisition failure.

In `@crates/vm/src/version.rs`:
- Around line 36-41: The format string in version.rs currently has an extra
space before the comma ("{} ,") causing a minor output typo; update the format!
call (the string passed to format! in the function that uses
get_version_number(), get_build_info(), pvm_version, and
env!("CARGO_PKG_VERSION")) to remove the space before the comma (change "{} ,"
to "{},") so the output reads "PVM VERSION {}, based on RustPython {} ..."
without the stray space.

In `@crates/vm/src/vm/checkpoint.rs`:
- Around line 209-219: Remove the two empty debug for-loops that iterate over
dict_items and varnames in checkpoint.rs: delete the loop over dict_items.iter()
that matches key.downcast_ref::<crate::builtins::PyStr>() and the subsequent
loop over varnames.iter() that checks locals_dict.get_item_opt(*varname, vm);
also remove the "Debug:" comment above the latter. Ensure no other logic
depended on these iterations (dict_items, varnames, locals_dict, vm) and run
cargo check to catch any unused-variable warnings after removal.
- Around line 468-477: Delete the unused helper ensure_supported_frame which
conflicts with multi-frame checkpoint logic: remove the entire function
definition (including the #[allow(dead_code)] attribute) and any dead imports
only used by it; keep validate_frame_for_checkpoint, VirtualMachine, and
FrameRef untouched so existing checkpoint logic that handles both single- and
multi-frame cases (the frame_refs.len() branching) remains correct.
- Around line 23-29: Uncomment and restore the frame validation loop in
save_checkpoint: re-enable the calls to validate_frame_for_checkpoint(vm, frame)
for each frame in frame_refs (the loop currently commented out) so
save_checkpoint matches other variants like
save_checkpoint_with_lasti_stack_blocks_and_locals; ensure the borrow on
vm.frames is dropped before running this loop (the explicit drop at line 22 is
present) and preserve the existing error propagation (the ? operator) from
validate_frame_for_checkpoint.

In `@crates/vm/src/vm/compile.rs`:
- Around line 73-75: resume_from_bytes currently calls
checkpoint::resume_script_from_bytes without performing the importer detection
or safe_path enforcement that run_script_resume applies; update
resume_from_bytes to mirror run_script_resume by (1) checking for an importer
for script_path (same logic used in run_script_resume between lines 54–60) and
returning an error if one exists, and (2) honoring the safe_path configuration
(same safe_path check used in run_script_resume between lines 61–71) before
calling checkpoint::resume_script_from_bytes; alternatively, if you prefer to
keep resume_from_bytes thin, document that callers must call ensure_sys_path()
and perform both the importer detection and safe_path checks (and update the
caller in crates/pvm-runtime/src/lib.rs to do so).

In `@examples/ast_visualize/ast_view.py`:
- Line 7: Remove the unused import by deleting "Iterable" from the typing import
statement in ast_view.py (the line "from typing import Iterable") so only
actually used types remain imported; ensure no other references to Iterable
exist in functions or classes such as any type annotations in ast_view.py before
committing.

In `@examples/ast_visualize/README.md`:
- Line 26: Update the README.md in examples/ast_visualize to remove
mixed-language text by replacing Chinese phrases with English equivalents:
change the heading "Graphviz 安装与渲染" to "Graphviz Installation & Rendering",
replace "安装完成后将 DOT 渲染为图片:" with "After installation, render DOT to an image:",
and replace "打开图片:" with "Open the image:"; scan the rest of README.md for the
same Chinese strings ("Graphviz 安装与渲染", "安装完成后将 DOT 渲染为图片:", "打开图片:") and update
them to the provided English phrases to ensure consistent language throughout
the document.

In `@examples/pvm_dex_demo/README.md`:
- Around line 14-18: Move the environment variable line into the fenced code
block so the entire multi-line command is copy-pastable: wrap the
DYLD_LIBRARY_PATH=/opt/homebrew/opt/libffi/lib \ line inside the same
triple-backtick bash block that contains the cargo run command (so the backslash
remains at the end of the environment line and the whole command is inside the
code fence).

In `@examples/pvm_runtime_chain_demo/escrow_marketplace_demo.py`:
- Around line 360-374: The loop currently calls _save_state after every action,
including read-only actions; change _run_actions (and the similar block in main)
to save state only when the action actually mutated state or is not read-only —
e.g., have _apply_action return a summary flag (summary.get("read_only") or
similar) or compare the returned state to the prior state and call _save_state
only when summary.get("read_only") is not True (or when prior_state !=
new_state). Update both _run_actions and the corresponding main save logic to
follow this check so GAS_READ-only actions do not trigger persistence.
- Around line 147-163: _handle_init currently calls _save_state which duplicates
a save that the caller also performs; remove the in-handler persistence to avoid
double writes and gas mischarging. Specifically, delete the _save_state(state)
call from _handle_init (or change _handle_init to return the new state instead
of persisting) and ensure the caller that invokes _handle_init (the outer
dispatch code that already persists after actions) performs the single save;
keep the state initialization logic (version, fee_bps, treasury, next_order_id,
balances, orders) intact in _handle_init.

In `@examples/pvm_runtime_chain_demo/fsm_demo.py`:
- Around line 12-17: The call to analyze(None, {}) inside main creates a
coroutine but never runs it; schedule the async continuation before returning by
either making main async and awaiting analyze(None, {}) or—preferably following
the pattern in checkpoint_demo.py—use the runner.run helper to start the
coroutine (so replace the discarded analyze call with runner.run(analyze(None,
{}))). Keep the existing continuation.new_cid and pvm_host.set_state usage
intact so cid is set before scheduling the coroutine.

In `@examples/pvm_runtime_chain_demo/main.rs`:
- Around line 113-143: The trace-handling block can re-enable determinism even
when the user passed --nondeterministic because options =
options.with_determinism(det) runs after you set options.deterministic = false
and options.determinism = None; change the logic so that
DeterminismOptions::deterministic(...) is only applied when the user did not
request nondeterministic mode — e.g., check the original deterministic flag (or
options.deterministic) before calling options.with_determinism(det), or move the
nondeterministic branch to run after the trace block and preserve
options.determinism = None; update the code around options, deterministic,
with_determinism, and DeterminismOptions::deterministic to ensure trace_path
cannot silently override a --nondeterministic setting and emit a warning if the
flags conflict.

In `@examples/pvm_runtime_chain_demo/staking_rewards_demo.py`:
- Around line 310-316: The code uses random.Random(seed_int) to select
sample_validators, which will fail if the global random module is blacklisted;
replace this with the deterministic pvm_random utilities (or add random to the
demo whitelist). Specifically, change the sampling logic around validator_names,
rng, sample_count and sample_validators to use pvm_random seeded with seed_int
(e.g., create a pvm_random.Random/PRNG instance or use pvm_random.sample) to
deterministically sample up to two validators, preserving the sorted order and
the fallback to an empty list when sample_count is zero.
- Around line 1-6: The demo imports the stdlib random module (import random) and
instantiates random.Random(seed_int), which will be blocked by the determinism
blacklist; replace the stdlib usage with the whitelisted deterministic RNG
(pvm_random) instead: remove the "import random" line, import the pvm-provided
RNG (e.g. from pvm_host import pvm_random or import pvm_random from the host
API) and change the instantiation random.Random(seed_int) to the pvm_random
equivalent (e.g. pvm_random.Random(seed_int)), preserving the deterministic seed
usage.
🧹 Nitpick comments (34)
examples/pvm_runtime_chain_demo/README.md (3)

1-14: Consider adding a Prerequisites section.

The README jumps directly into file descriptions without mentioning what needs to be installed or configured. Adding a brief Prerequisites section before "Files" would help users prepare their environment.

For example:

  • Rust toolchain version
  • Python version requirements
  • libffi (especially for macOS users)
  • Any other dependencies

23-28: Consider consolidating macOS libffi instructions.

The DYLD_LIBRARY_PATH setup for macOS appears three times (lines 23-28, 39-40, 102-106). While this repetition ensures users don't miss it, you could improve organization by:

  1. Creating a "Troubleshooting" or "Platform-Specific Notes" section, or
  2. Moving this to a Prerequisites section with a note like "macOS users: Export this before running any commands"

This would reduce duplication while maintaining clarity.

Also applies to: 39-40, 102-106


109-113: Clarify scope of "Contract Behavior" section.

This section describes only the basic contract.py behavior, not the other demo contracts (escrow_marketplace_demo.py, batch_payroll_demo.py, staking_rewards_demo.py) listed in the Files section. Consider either:

  1. Renaming to "Basic Contract Behavior" to clarify the scope, or
  2. Expanding to briefly describe what each demo contract demonstrates

This would help users understand what to expect when running the different demos.

examples/ast_visualize/README.md (1)

10-12: Consider adding language hints to fenced code blocks.

Multiple code blocks lack language specification, which affects syntax highlighting and tooling. For shell commands, use bash or sh; for the command examples, use bash.

Example fix for a few blocks
 Tree view (default):

-```
+```bash
 ./target/release/rustpython examples/ast_visualize/ast_view.py --file examples/ast_visualize/sample.py

...

macOS (Homebrew):

- +bash
brew install graphviz

Also applies to: 16-18, 22-24, 30-32, 36-38, 42-45, 49-51, 55-57, 61-63, 67-69, 73-75, 81-83, 87-89, 93-95

examples/ast_visualize/ast_view.py (1)

150-151: Consider escaping additional DOT special characters.

The current escaping handles backslashes and double quotes, but DOT labels may also need escaping for newlines and potentially angle brackets if they appear in node summaries (e.g., from string literals containing < or >).

More comprehensive escaping
 def escape_dot_label(text: str) -> str:
-    return text.replace("\\", "\\\\").replace('"', "\\\"")
+    return (
+        text.replace("\\", "\\\\")
+        .replace('"', '\\"')
+        .replace("\n", "\\n")
+        .replace("<", "\\<")
+        .replace(">", "\\>")
+    )
examples/pvm_runtime_chain_demo/run_checkpoint_demo.sh (1)

24-30: Backup filenames can collide within the same second.

If the script runs twice quickly, date +%s can produce the same suffix and mv will fail. Consider adding PID or using mktemp for uniqueness.

💡 Example tweak
-ts=$(date +%s)
+ts="$(date +%s)-$$"
examples/breakpoint_resume_demo/state_store.py (1)

19-22: Consider atomic writes for state persistence.

Direct writes can leave partial/corrupt JSON if interrupted.

🛡️ Suggested refactor (atomic write)
-import json
+import json
+import os
+import tempfile
 from pathlib import Path
 from typing import Any
@@
 def save_state(state: dict[str, Any]) -> None:
     # Write checkpoint state, ensuring the directory exists.
     DATA_DIR.mkdir(parents=True, exist_ok=True)
-    STATE_FILE.write_text(json.dumps(state, sort_keys=True, indent=2))
+    payload = json.dumps(state, sort_keys=True, indent=2)
+    with tempfile.NamedTemporaryFile(
+        "w", delete=False, dir=DATA_DIR, encoding="utf-8"
+    ) as tmp:
+        tmp.write(payload)
+        tmp_path = tmp.name
+    os.replace(tmp_path, STATE_FILE)
examples/pvm_actor_transfer_demo/main.rs (1)

129-137: Optional: de-duplicate encode_hex.

This helper matches the one in examples/pvm_runtime_chain_demo/main.rs (lines 149-157); consider a shared utility to reduce drift.

examples/pvm_actor_transfer_demo/contract.py (2)

41-44: bool values pass the int check due to Python's type hierarchy.

In Python, bool is a subclass of int, so isinstance(True, int) returns True. If boolean values should be rejected, add an explicit check.

♻️ Proposed fix to exclude booleans
 def _require_int(value, name):
-    if not isinstance(value, int):
+    if not isinstance(value, int) or isinstance(value, bool):
         raise ValueError(f"{name} must be int")
     return value

140-151: Consider adding authorization for mint operations.

The _handle_mint function allows anyone to mint tokens without authorization checks. While this may be intentional for a demo, in production smart contracts, minting should typically be restricted to authorized accounts.

examples/breakpoint_resume_demo/comprehensive_demo.py (1)

77-77: Remove unnecessary f-string prefix.

Multiple print statements use f-strings without placeholders. Per PEP 8 and linting guidelines, use regular strings when there are no interpolated values.

♻️ Proposed fix for f-strings without placeholders
-print(f"\n[Test 2] For Loop with List Iteration")
+print("\n[Test 2] For Loop with List Iteration")

Similar changes apply to lines: 88, 96, 107, 115, 127, 135, 150, 161, 168, 171, 181, 191, 203, 211, 224, 235, 241, 244, 252, 267, 270, 278, 289, 292, 300, 313, 316, 324, 337, 340, 357, 363, 365, 370.

examples/breakpoint_resume_demo/test_checkpoint.py (2)

6-6: Remove unused sys import.

The sys module is imported but never used. The script uses raise SystemExit(main()) which doesn't require sys.

♻️ Proposed fix
-import sys

10-17: Binary name mismatch in help text.

The resolve_bin_path function looks for pvm binary, but the --bin argument help text says "rustpython executable". Consider aligning the documentation with the actual binary name.

♻️ Proposed fix
     parser.add_argument(
         "--bin",
-        help="Path to the rustpython executable, defaults to target/release/rustpython",
+        help="Path to the pvm executable, defaults to target/release/pvm",
     )

Also applies to: 29-33

examples/pvm_runtime_chain_demo/determinism_check.py (2)

55-58: Consider using a more explicit deduplication approach.

Using dict.fromkeys(paths) for deduplication works but is somewhat indirect. A more readable approach would use a set or explicit loop for clarity.

♻️ Suggested refactor
     if paths:
-        env["DYLD_FALLBACK_LIBRARY_PATH"] = os.pathsep.join(
-            dict.fromkeys(paths)
-        )
+        # Deduplicate while preserving order
+        seen = set()
+        unique_paths = [p for p in paths if not (p in seen or seen.add(p))]
+        env["DYLD_FALLBACK_LIBRARY_PATH"] = os.pathsep.join(unique_paths)

166-171: Narrow the exception catch for better error diagnostics.

Catching bare Exception masks the specific failure mode. Consider catching the expected exceptions (ValueError from fromhex, UnicodeDecodeError from decode, json.JSONDecodeError from loads).

♻️ Suggested refactor
         if args.decode:
             try:
                 decoded = bytes.fromhex(out_hex).decode("utf-8")
                 parsed = json.loads(decoded)
                 print(json.dumps(parsed, indent=2, sort_keys=True))
-            except Exception as exc:
+            except (ValueError, UnicodeDecodeError, json.JSONDecodeError) as exc:
                 print(f"decode failed: {exc}")
examples/pvm_runtime_chain_demo/determinism_demo.py (1)

39-43: Use a context manager for the file write.
This avoids leaving the file handle open when the write succeeds.

♻️ Suggested tweak
-    open("forbidden.txt", "wb").write(b"x")
+    with open("forbidden.txt", "wb") as f:
+        f.write(b"x")
examples/pvm_alto_call_demo.rs (1)

41-44: Consider adding explicit path validation and configurability for better error messages.

The contract file at examples/pvm_actor_transfer_demo/contract.py currently exists, and the ? operator provides appropriate error handling. However, an explicit existence check would provide a clearer error message if the file is ever missing or renamed:

Optional improvement for fail-fast with clearer error
-    let script_path = "examples/pvm_actor_transfer_demo/contract.py";
-    let code = fs::read(script_path)?;
+    let script_path = PathBuf::from("examples/pvm_actor_transfer_demo/contract.py");
+    if !script_path.exists() {
+        return Err(format!("missing contract script: {}", script_path.display()).into());
+    }
+    let code = fs::read(&script_path)?;
-    let options = default_options().with_source_path(script_path);
+    let options = default_options().with_source_path(script_path.to_string_lossy());
crates/vm/src/stdlib/rustpython_checkpoint.rs (1)

22-34: Consider factoring shared checkpoint enqueue logic.

checkpoint and checkpoint_bytes duplicate the same frame/lasti/request logic; a small helper would keep the error path and locking behavior consistent.

crates/vm/src/vm/compile.rs (1)

54-70: Avoid drift by sharing the safe-path/sys.path logic.

run_script and run_script_resume now duplicate the same directory insertion; consider extracting a small helper to keep behavior aligned.

examples/pvm_runtime_chain_demo/main.rs (1)

22-82: Treat unknown flags as errors to avoid mis-parsing script path.

Right now any unrecognized --flag is accepted as a script path, which can produce confusing errors. Consider rejecting unknown options that start with -.

♻️ Suggested tweak
             _ => {
+                if arg.starts_with('-') {
+                    return Err(usage().into());
+                }
                 if script_path.is_none() {
                     script_path = Some(arg);
                 } else if input.is_none() {
                     input = Some(arg.into_bytes());
                 } else {
examples/pvm_runtime_chain_demo/checkpoint_demo.py (1)

5-6: Handle immediate coroutine completion in run.

If the coroutine returns without awaiting, send(None) raises StopIteration; catching it makes the helper more robust.

♻️ Suggested tweak
 def run(coro):
-    return coro.send(None)
+    try:
+        return coro.send(None)
+    except StopIteration as exc:
+        return exc.value
src/settings.rs (1)

157-163: Consider validating --resume/--continuation pairing.
If one flag is supplied without the other, consider defaulting or emitting a clear error to avoid ambiguous behavior.

examples/pvm_dex_demo/main.rs (2)

19-80: Consider using clap for argument parsing.

The manual argument parsing is functional but verbose. For an example binary, this is acceptable to minimize dependencies. However, if this example grows, consider using the clap crate for more robust and maintainable CLI handling.


120-128: Duplicate encode_hex implementation.

This function is identical to the one in crates/pvm-alto/src/lib.rs. Consider either:

  1. Exporting encode_hex from pvm_alto and using it here, or
  2. Adding a shared utility crate for common helpers.

For an example binary, some duplication is tolerable, but consolidating would improve maintainability.

crates/codegen/src/pvm_fsm.rs (3)

228-263: Consider handling multiple return statements.

The parse_body function (lines 228-263) processes the body sequentially and captures the last return statement's expression. If multiple return statements exist, only the last one is used for the generated FSM, which could lead to unexpected behavior.

Consider either:

  1. Validating that only one return statement exists at the end, or
  2. Documenting this limitation clearly.

396-476: Generated code uses inline imports.

The generated FSM functions include import pvm_sdk inside each function body (lines 409, 430). While this ensures the import is available regardless of module-level imports, it incurs repeated import overhead.

This is acceptable for the code generation approach, but consider documenting that generated functions have this characteristic or exploring module-level import tracking if performance becomes a concern.


478-503: Similar contains_await pattern exists in compile.rs.

This contains_await implementation is similar to the one in crates/codegen/src/compile.rs (lines 5764-5780), but operates on Stmt instead of Expr. Both are needed for their respective contexts, but consider extracting a shared visitor utility if more await-detection logic is needed elsewhere.

crates/pvm-runtime/src/module.rs (1)

118-152: Consider using specialized exception types in host_error.

DeterministicValidationError, NonDeterministicError, and OutOfGasError are defined but the host_error function always returns the base HostError type. Consider mapping specific HostError variants to these specialized types for better Python-side exception handling.

♻️ Suggested enhancement
 fn host_error(vm: &VirtualMachine, err: HostError) -> PyBaseExceptionRef {
-    let exc = vm.new_exception(
-        host_error_type(vm),
-        vec![vm.ctx.new_str(err.to_string()).into()],
-    );
+    let exc_type = match &err {
+        HostError::OutOfGas => out_of_gas_error_type(vm),
+        HostError::Forbidden => nondeterministic_error_type(vm),
+        HostError::InvalidInput => deterministic_validation_error_type(vm),
+        _ => host_error_type(vm),
+    };
+    let exc = vm.new_exception(
+        exc_type,
+        vec![vm.ctx.new_str(err.to_string()).into()],
+    );
     let _ = exc
         .as_object()
         .set_attr("code", vm.new_pyobj(err.code()), vm);
crates/vm/src/frame.rs (1)

568-571: SystemExit after checkpoint may confuse error handling.

After successfully saving a checkpoint, raising SystemExit with message "checkpoint exit" could be confusing since it's not an error. The calling code in crates/pvm-runtime/src/lib.rs handles this by checking for checkpoint bytes, but a more explicit signal (like a custom exception type) would be clearer.

crates/pvm-runtime/src/lib.rs (3)

164-190: Modifying determinism options during resume changes behavior mid-execution.

The code modifies the whitelist/blacklist of the determinism options when resume_bytes is present to allow OS-related imports. This mutation of the options struct could lead to unexpected behavior if the options are reused. Consider cloning or documenting this side effect.


319-327: Using expect for dict set operations could panic.

While these operations are unlikely to fail, using expect in library code can cause unexpected panics. Consider propagating errors or logging warnings instead.

♻️ Suggested change
     main_module
         .dict()
         .set_item("__annotations__", vm.ctx.new_dict().into(), vm)
-        .expect("Failed to initialize __main__.__annotations__");
+        .ok(); // Ignore errors for optional metadata
     main_module
         .dict()
         .set_item("__file__", vm.ctx.new_str(options.source_path.clone()).into(), vm)
-        .expect("Failed to initialize __main__.__file__");
+        .ok();
     main_module
         .dict()
         .set_item("__cached__", vm.ctx.none(), vm)
-        .expect("Failed to initialize __main__.__cached__");
+        .ok();

562-589: Consider using serde_json for JSON serialization if comprehensive JSON compliance is needed.

The manual JSON escaping implementation currently handles basic cases (\, ", \n, \r, \t) but misses escaping control characters that JSON specification requires (e.g., backspace \b, form feed \f, and other ASCII control characters below 0x20). If serde_json is acceptable as a dependency, it would provide more robust and spec-compliant serialization.

crates/vm/src/vm/checkpoint.rs (2)

232-239: Empty else branch in fastlocals restoration loop.

The else branch is empty and serves no purpose. Consider removing it for clarity.

♻️ Suggested fix
         for (idx, varname) in varnames.iter().enumerate() {
             if let Some(value) = locals_dict.get_item_opt(*varname, vm)? {
                 fastlocals[idx] = Some(value);
-            } else {
             }
         }

433-436: Remove redundant emptiness check.

The check if !frames.is_empty() on line 434 is redundant since line 423-425 already returns an error if frames is empty.

♻️ Suggested fix
     // Build blocks vec: only innermost frame gets blocks, others get empty vec.
     // Outer frames are waiting for inner frames to return and their block state
     // can be safely reconstructed as empty since they're not in active control flow.
     let mut all_blocks = vec![Vec::new(); frames.len()];
-    if !frames.is_empty() {
-        all_blocks[frames.len() - 1] = innermost_blocks;
-    }
+    all_blocks[frames.len() - 1] = innermost_blocks;

10 hidden conversations Load more…
@youknowone youknowone marked this pull request as draft January 18, 2026 03:41

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Thank you for patch.

Because this is a huge patch. I need to understand it better. Before starting review, here is a checklist for big patch.

  • Please describe the motivation of the patch.
  • Please ensure pvm is a hard dependency of this feature.
  • This is not a core Python feature. It must be optional
  • Is this the best minimal size of the patch? Usually merging a huge single patch is not easy. Please break it down to smaller ones if possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants