Run benchmarks with stdlib native modules by moreal · Pull Request #6792 · RustPython/RustPython

moreal

Conversation

moreal commented
Jan 18, 2026
•

edited by coderabbitai Bot

Loading

Personally, I was checking https://rustpython.github.io/benchmarks to see if benchmarks improved after the json.loads enhancement, but there was actually no performance improvement. Upon reviewing this again, I suspect it was running with a Pure Python implementation using an interpreter without stdlib. So if there are no particular issues, how about running the benchmark with a version that includes stdlib?

I'm not very familiar with whether the presence of a standard library has a significant impact on performance benchmarks, so I would be really happy if you could share your opinion!

Summary by CodeRabbit

Chores
- Added standard library support to development dependencies to enable richer test environments.
- Updated development benchmarks to initialize with the standard library, yielding more realistic performance measurements.
- No changes to public/external APIs or exported signatures.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

📝 Walkthrough

Walkthrough

Adds rustpython-stdlib as a dev-dependency and changes benchmark VM construction to initialize and register RustPython native stdlib modules before running benchmarks.

Changes

Cohort / File(s)	Summary
Dependency Addition `\`Cargo.toml``	Added `rustpython-stdlib` to `[dev-dependencies]`.
Benchmark VM Initialization `\`benches/execution.rs``	Switched from `Interpreter::without_stdlib` to `Interpreter::with_init` and added `vm.add_native_modules(rustpython_stdlib::get_module_inits())` in the VM pre-enter initialization block.

Sequence Diagram(s)

sequenceDiagram
  participant Bench as Bench Harness
  participant Init as Interpreter::with_init
  participant VM as VM (pre-enter block)
  participant Stdlib as rustpython_stdlib

  Bench->>Init: construct Interpreter::with_init(...)
  Init->>VM: enter pre-init closure
  VM->>Stdlib: rustpython_stdlib::get_module_inits()
  VM->>Stdlib: vm.add_native_modules(module_inits)
  VM-->>Bench: VM ready with native stdlib modules
  Bench->>VM: run benchmark code (imports/use stdlib)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

Introduce json_loads executions benchmark #6723: Updates benchmark harness to initialize and register rustpython stdlib native modules, directly related to enabling stdlib-dependent benchmarks.

Suggested reviewers

youknowone

Poem

🐰
I hopped into Cargo with a cheerful nod,
Brought stdlib seeds to the benchmark sod.
Native modules sown before the run,
Now imports dance beneath the sun. ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Run benchmarks with stdlib native modules' directly and clearly describes the main change: configuring benchmarks to use the standard library with native modules instead of running without it.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Code has been automatically formatted

The code in this PR has been formatted using:

cargo fmt --all
Please pull the latest changes before pushing again:

git pull origin benchmark-with-stdlib

Change benchmark interpreter from without_stdlib to with_init to load
Rust-optimized native modules like _json, ensuring benchmarks measure
actual optimized performance rather than pure Python fallbacks.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

moreal marked this pull request as draft