◐ Shell
clean mode source ↗

bpo-29585: optimize site.py startup time by methane · Pull Request #136 · python/cpython

@methane methane changed the title optimize site.py [RFC] bpo-29585: optimize site.py startup time

Feb 17, 2017

vstinner

@methane methane changed the title [RFC] bpo-29585: optimize site.py startup time [RFC] [bpo-29585](https://bugs.python.org/issue29585): optimize site.py startup time

Feb 18, 2017

@methane methane changed the title [RFC] [bpo-29585](https://bugs.python.org/issue29585): optimize site.py startup time [RFC] bpo-29585: optimize site.py startup time

Feb 18, 2017

@methane methane changed the title [RFC] bpo-29585: optimize site.py startup time bpo-29585: optimize site.py startup time

Feb 19, 2017
Skip importing sysconfig when possible.

Median +- std dev: [default] 15.8 ms +- 0.0 ms -> [patched] 14.7 ms +- 0.0 ms: 1.07x faster (-7%)

@methane

@methane

vstinner

@methane

@methane

vstinner

vstinner

@methane methane deleted the optimize-site-startup branch

June 28, 2017 15:31

vstinner added a commit that referenced this pull request

Jun 28, 2017

akruis pushed a commit to akruis/cpython that referenced this pull request

Oct 10, 2017

akruis pushed a commit to akruis/cpython that referenced this pull request

Oct 29, 2017

SonicField added a commit to SonicField/cpython that referenced this pull request

Apr 25, 2026
…_inline_except_opcode_array_c

Fixes THREE boundary-domain bugs in build_inline_except_opcode_array_c
(introduced W27c #2a 7135d94), found across two HIR-diff Phase 0 cycles
of the W-2B-RECONVERT investigation.

CONVENTION (Python/jit/bytecode.cpp:8-14, builder.cpp:1235,
phx_frame_state.h cur_instr_offs semantics):
- jit_bc_instr_init expects INSTRUCTION INDEX (codeUnit[])
- jit_bc_instr_get_jump_target / next_offset / base_offset return whatever
  was stored at init (now INDEX after Class A fix)
- phx_block_map keys are BYTE OFFSETS
- OpcodeArrayEntry.base_offset is consumed downstream as BYTE OFFSET
  (cur_instr_offs assignment per builder_emit_c.c:3320)
- BCOffset.value() (caller-passed except_body_offset) is BYTE OFFSET
- BYTES = INDEX * sizeof(_Py_CODEUNIT) (= 2 in 3.12)

NAMED CONVERSIONS (Python/jit/bytecode_c.h, gated by _Py_OPCODE):
  static inline int phx_bc_offset_to_instr_index(int byte_off);
  static inline int phx_bc_instr_index_to_offset(int instr_idx);

Codifies the boundary-domain rule by example (per pythia python#137 python#2 +
supervisor 19:01:19Z + theologian 19:01:17Z).

THREE FIXES:

CLASS A (line 3241): jit_bc_instr_init was passed except_body_offset
(BCOffset.value() byte offset) where INSTRUCTION INDEX was expected.
codeUnit(code)[byte_offset] read PAST end of co_code → garbage opcode →
switch-default → Deopt with corrupt frame state. Found by Phase 0
HIR-diff (test_exc_raise_catch bb 12: correct Return -1 vs corrupt
LoadConst NoneType + Deopt at offset 58).

CLASS B (line 3273-3275, theologian class-of-bug audit 18:42:53Z):
target = jit_bc_instr_get_jump_target returns INDEX, but
phx_block_map_lookup_or_panic expects BYTE OFFSET. Without conversion,
JUMP_BACKWARD-in-except-body lookup fails: JIT_CHECK_C panic OR silent
wrong-block. Dormant pre-fix because no test had backward-jump-in-except-body.

CLASS C (line 3260, exposed by Phase 0' HIR-diff after Class A+B fix):
After Class A fix corrected the init to INDEX, jit_bc_instr_base_offset
returns INDEX. But entry->base_offset is consumed downstream (line 3320)
as BYTE OFFSET via match_tc.frame.cur_instr_offs assignment. Pre-fix
'correct by accident' — Class A's BYTES-as-INDEX init wrote BYTES into
bci->base_offset, so jit_bc_instr_base_offset returned BYTES, matching
downstream. Correct Class A exposed Class C: cur_instr_offs got INDEX
(half the correct BYTE value) → interpreter Deopt resumed at wrong
bytecode position → SIGSEGV in test_multiple_exceptions_in_loop
(deterministic 0/20 post Class A+B fix, vs 20/20 PASS pre-W27c).

DIAGNOSIS:
HIR-diff for test_multiple_exceptions_in_loop revealed Deopt CurInstrOffset
124 (correct, BYTES) → 62 (wrong, INDEX = 124/2). Direct evidence of the
domain mismatch.

LATENT in pushed W27c #2a (e4e7507 on SonicField/cpython): all three
classes present. Class A, B dormant (no test exercises emitInlineExceptionMatch
or JUMP_BACKWARD-in-except-body). Class C compensated by Class A — both
broken in opposite directions canceling out for downstream consumers of
entry->base_offset. ALL three must fix together.

INVESTIGATION CHAIN:
- testkeeper bisect 17:52:30Z localized #2b regression → W27c #2b sole
- pythia python#136 python#1 18:23:34Z flagged HEAP/RACE rode on absence-of-evidence
- generalist 18:24Z proposed HIR-diff Phase 0 falsifier
- generalist 18:32+18:34Z captured HIR_2a + HIR_2b dumps; found Class A
- theologian 18:42:53Z class-of-bug audit found Class B
- supervisor 18:43:17Z directed dual-fix
- pythia python#137 python#2 19:00:29Z flagged inline-arithmetic violates boundary-domain rule
- supervisor 19:01:19Z + theologian 19:01:17Z directed amend to named conversions
- testkeeper 19:06:55Z full Phoenix gate caught NEW regression
  (test_multiple_exceptions_in_loop deterministic 0/20)
- generalist 19:14Z HIR-diff Phase 0' on test_multiple_exceptions_in_loop
  revealed Class C (cur_instr_offs 124→62)
- supervisor 19:16:42Z authorized Class C fix + extended audit

OTHER OpcodeArrayEntry FIELDS AUDITED (per supervisor 19:16:42Z extended
class-of-bug discipline):
- entry->opcode: written from jit_bc_instr_opcode (no domain — opcode value);
  consumed in dispatch loop switch. CLEAN.
- entry->oparg: written from jit_bc_instr_oparg (no domain — oparg value);
  consumed in dispatch loop emit calls. CLEAN.
- entry->base_offset: Class C above; FIXED.
- entry->const_obj: written from PyTuple_GET_ITEM (PyObject*); consumed in
  dispatch loop hir_type_from_object. CLEAN (no domain conversion).
- entry->jump_target_block: Class B above; FIXED.

VERIFICATION pending (testkeeper 4-suite extended verify):
1. 30x test_exc_raise_catch (Class A regression)
2. 30x test_exc_binary_subscr_dict_in_try (Class A latent activation)
3. 30x test_exc_continue_in_loop (Class B latent activation)
4. 30x multi-except-in-loop sentinel (Class C latent activation)
5. Full Phoenix suite (which originally caught Class C)