◐ Shell
reader mode source ↗
Skip to content

gh-146442: Fix various bugs in compiler pipeline from bug report#146443

Merged
vstinner merged 4 commits into
python:mainfrom
A0su:fix_error_handling_and_ref_leaks_in_compiler
Mar 30, 2026
Merged

gh-146442: Fix various bugs in compiler pipeline from bug report#146443
vstinner merged 4 commits into
python:mainfrom
A0su:fix_error_handling_and_ref_leaks_in_compiler

Conversation

@A0su

@A0su A0su commented Mar 26, 2026 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Contributor

Compiler Pipeline (codegen + compile + symtable + flowgraph + assemble) | 6 FIX

codegen.c (6,632 lines) — 4 FIX

  1. NULL deref + ref leak in codegen_setup_annotations_scope (line 716-720): PyLong_FromLong result not NULL-checked, passed to ADDOP_LOAD_CONST which dereferences it. Also never DECREF'd even on success.
  2. Ref leak of mangled in codegen_nameop (line 3280): RETURN_IF_ERROR(scope) early-returns without DECREF'ing mangled from _PyCompile_MaybeMangle.
  3. Unchecked _PyCompile_PushFBlock in codegen_unwind_fblock_stack (line 669): Return value silently discarded, error lost, frame block stack left inconsistent.

compile.c (1,772 lines) — 1 FIX

  1. Ref leak of orig in _PyCompile_TweakInlinedComprehensionScopes (line 1096-1114): PyDict_GetItemRef returns new ref in orig, never DECREF'd on any path (success or error). Leaks on every inlined comprehension.

assemble.c (802 lines) — 1 FIX

  1. assemble_emit_instr returns ERROR without exception (line 420-421): Bytecode overflow check returns ERROR without PyErr_NoMemory() → SystemError.

symtable.c (3,355 lines) — 0 FIX (clean)

flowgraph.c (4,088 lines) — 0 FIX (clean)

Did not fix

  1. Ref leak of mangled in codegen_deferred_annotations_body (line 792-818)
    • I do not see a simple way to fix this without overhauling the macros (ADDOP* & VISIT) that can early return

@bedevere-app bedevere-app Bot mentioned this pull request Mar 26, 2026
Open
@brianschubert brianschubert changed the title gh-146102: Fix various bugs in compiler pipeline from bug report Mar 27, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Mar 27, 2026
Closed
vstinner
vstinner approved these changes Mar 30, 2026
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

LGTM

        // NOTE: ref of mangled can be leaked on ADDOP* and VISIT macros due to early returns
        // fixing would require an overhaul of these macros

That's unfortunate, but this change already fix multiple issues and is worth it. The issues described in the comment can be fixed in a follow-up change is someone is interested.

@bedevere-app bedevere-app Bot added and removed awaiting review labels Mar 30, 2026
Hide details View details @vstinner vstinner merged commit ca95e97 into python:main Mar 30, 2026
53 checks passed
@vstinner

vstinner commented Mar 30, 2026

Copy link
Copy Markdown
Member

Merged, thanks for your fixes.

@A0su

A0su commented Mar 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

LGTM

        // NOTE: ref of mangled can be leaked on ADDOP* and VISIT macros due to early returns
        // fixing would require an overhaul of these macros

That's unfortunate, but this change already fix multiple issues and is worth it. The issues described in the comment can be fixed in a follow-up change is someone is interested.

Would the preferred approach here be to introduce the same macros but accepting an object to DecRef. Similar to how some of the other existing do?

@vstinner

vstinner commented Mar 30, 2026

Copy link
Copy Markdown
Member

You may move to a subfunction and DECREF(mangled) if the subfunction fails.

@A0su

A0su commented Mar 31, 2026

Copy link
Copy Markdown
Contributor Author

Ah ok, I can raise the issue and work on the refactor over the next couple days.

maurycy added a commit to maurycy/cpython that referenced this pull request Mar 31, 2026
@maurycy
Merge branch 'main' into tachyon-ndjson-kolektor …
9836ffa 
* main:
  pythongh-145458: use `self.skip_idle` consistently in the tachyon profiler (python#145459)
  pythongh-146615: Fix format specifiers in Objects/ directory (pythonGH-146620)
  pythongh-146615: Fix format specifiers in Python/ directory (pythonGH-146619)
  pythongh-146615: Fix format specifiers in test cextensions (pythonGH-146618)
  pythongh-146615: Fix format specifiers in extension modules (pythonGH-146617)
  pythongh-146615: Fix crash in __get__() for METH_METHOD descriptors with invalid type argument (pythonGH-146634)
  pythongh-146376: Reduce timeout in Emscripten GHA workflow (python#146378)
  pythongh-146442: Fix various bugs in compiler pipeline (python#146443)
  pythongh-146238: Support half-floats in the array module (python#146242)
  pythongh-145056: Add support for merging collections.UserDict and frozendict (pythonGH-146465)
  pythongh-145056: Fix merging of collections.OrderedDict and frozendict (pythonGH-146466)
  pythongh-139633: Run netrc file permission check only once per parse (pythonGH-139634)
@A0su A0su mentioned this pull request Apr 3, 2026
Open
@pablogsal pablogsal mentioned this pull request Apr 4, 2026
Merged
clin1234 pushed a commit to clin1234/cpython that referenced this pull request Apr 16, 2026
@A0su @clin1234
pythongh-146442: Fix various bugs in compiler pipeline (python#146443) …
a82862c 
Fix null derefs, missing decrefs, and unchecked returns from bug report.
ljfp pushed a commit to ljfp/cpython that referenced this pull request Apr 25, 2026
@A0su @ljfp
pythongh-146442: Fix various bugs in compiler pipeline (python#146443) …
edb1b9b 
Fix null derefs, missing decrefs, and unchecked returns from bug report.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes

@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

@iritkatriel iritkatriel Awaiting requested review from iritkatriel iritkatriel is a code owner

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@A0su @vstinner