◐ Shell
reader mode source ↗
Skip to content

bpo-46409: Make generators in bytecode#30633

Merged
markshannon merged 20 commits into
python:mainfrom
faster-cpython:make-gen-in-bytecode
Jan 20, 2022
Merged

bpo-46409: Make generators in bytecode#30633
markshannon merged 20 commits into
python:mainfrom
faster-cpython:make-gen-in-bytecode

Conversation

@markshannon

@markshannon markshannon commented Jan 17, 2022

Copy link
Copy Markdown
Member

Adds a RETURN_GENERATOR bytecode which makes a generator (or coroutine) from the current frame.
This has a number of advantages:

  1. Generator frames cannot be incomplete, as the frame is fully initialized before creating the generator
  2. It simplifies the code for calling a Python function, as there is no need for a special case for generator functions.
  3. It removes the need to specialize calls to generator functions, they can be specialized as any other call to a Python function.

This PR also adds a JUMP_NO_INTERRUPT bytecode. This is the same as JUMP_ABSOLUTE without any checks for interrupts. Checking for interrupts in yield from would destroy the flaky pretense that Python coroutines are stackful. (They are not, but PEPs 380 and 489 like to pretend that they are), so we need the extra, seemingly redundant instruction.

We use RETURN_GENERATOR rather than MAKE_GENERATOR; RETURN_VALUE as the RETURN_VALUE would make the compiler treat all the following code as unreachable. We could fix that using artificial instructions, but I think that is best left to another PR.

https://bugs.python.org/issue46409

@markshannon

Copy link
Copy Markdown
Member Author

Fixes https://bugs.python.org/issue46389 and https://bugs.python.org/issue46374, but does not include tests for those issues.
I'll add the tests in separate PRs to close those issues.

@markshannon markshannon added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jan 17, 2022
@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit c33abbf 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jan 17, 2022
@markshannon markshannon added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jan 18, 2022
@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 243e441 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jan 18, 2022

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

I'm still trying to follow the rest of the C code here, but in case I fail, here's the rest of my review. :-)

@gvanrossum

Copy link
Copy Markdown
Member

PS. What do you mean by "coroutines are stackful"? (I was hoping that reading the code would clarify this, but either I didn't get to that point in the code yet or it's hidden by details.)

20 hidden items Load more…
@markshannon

Copy link
Copy Markdown
Member Author

PS. What do you mean by "coroutines are stackful"? (I was hoping that reading the code would clarify this, but either I didn't get to that point in the code yet or it's hidden by details.)

Stackful coroutines have their own stack, like Lua coroutines or greenlets.
async def coroutines are "stackless", since they do not have their own stack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants