◐ Shell
reader mode source ↗
Skip to content

bpo-26552: Fixed case where failing asyncio.ensure_future did not close the coroutine#30288

Merged
gvanrossum merged 2 commits into
python:mainfrom
kumaraditya303:issue26552
Jan 28, 2022
Merged

bpo-26552: Fixed case where failing asyncio.ensure_future did not close the coroutine#30288
gvanrossum merged 2 commits into
python:mainfrom
kumaraditya303:issue26552

Conversation

@kumaraditya303

@kumaraditya303 kumaraditya303 commented Dec 29, 2021 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Contributor

@kumaraditya303 kumaraditya303 changed the title bpo-26552: fix failing ensure_future which created a Task Dec 29, 2021
@carlosdmz

carlosdmz commented Dec 29, 2021 •
edited by gvanrossum
Loading

Copy link
Copy Markdown
Contributor

In Lib/asyncio/tasks.py, there's an _enclose_future()_ensure_future(), you might be able to find out whether the function has called wrap_awaitable or not. I couldn't get the edge case when there's a task with a loop closed, of which was stated on the bug report. You might be able to flag out the specific edge case and then call create_task with that flag and corner it.

@kumaraditya303

kumaraditya303 commented Jan 24, 2022

Copy link
Copy Markdown
Contributor Author

(Rebased to main)

@arhadthedev

arhadthedev commented Jan 24, 2022

Copy link
Copy Markdown
Member

*with hope* Does it fix test_asyncio random failures on a Windows runner so it can be enabled back?

@kumaraditya303

kumaraditya303 commented Jan 26, 2022

Copy link
Copy Markdown
Contributor Author

with hope Does it fix test_asyncio random failures on a Windows runner so it can be enabled back?

I fixed that issue separately.

gvanrossum
gvanrossum approved these changes Jan 28, 2022
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Hi Kumar, I had procrastinated on reviewing this because there's some complicated stuff going on. I finally found the time and noticed that the bpo issue was written somewhat misleadingly, so I've added a clarification there.

I now finally figured out what the real issue is and how to deal with it, and your code is correct:

  • If the argument is a Future, nothing more needs to be done
  • If it is a coroutine (i.e. async def), we need to close it if create_task() raises RuntimeError (which it only does for one reason, i.e. if the event loop is closed)
  • If it is an awaitable (i.e. something with an __await__() method), it may not have a close() method; the wrapper created by _wrap_awaitable() doesn't need to be closed

So I'm going to land this now. Thanks for your patience!

@gvanrossum gvanrossum merged commit 24cc641 into python:main Jan 28, 2022
@bedevere-bot

bedevere-bot commented Jan 28, 2022

Copy link
Copy Markdown

@gvanrossum: Please replace # with GH- in the commit message next time. Thanks!

@bedevere-bot

bedevere-bot commented Jan 28, 2022 •
edited
Loading

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE CentOS9 3.x has failed when building commit 24cc641.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/852/builds/10) and take a look at the build logs.
  4. Check if the failure is related to this commit (24cc641) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/852/builds/10

Summary of the results of the build (if available):

Click to see traceback logs 
remote: Enumerating objects: 13, done.        
remote: Counting objects:   9% (1/11)        
remote: Counting objects:  18% (2/11)        
remote: Counting objects:  27% (3/11)        
remote: Counting objects:  36% (4/11)        
remote: Counting objects:  45% (5/11)        
remote: Counting objects:  54% (6/11)        
remote: Counting objects:  63% (7/11)        
remote: Counting objects:  72% (8/11)        
remote: Counting objects:  81% (9/11)        
remote: Counting objects:  90% (10/11)        
remote: Counting objects: 100% (11/11)        
remote: Counting objects: 100% (11/11), done.        
remote: Total 13 (delta 11), reused 11 (delta 11), pack-reused 2        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '24cc6411adbfe5555ecd8901f1ea50caa414c908'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 24cc6411ad [bpo-26552](https://bugs.python.org/issue26552): Fixed case where failing `asyncio.ensure_future` did not close the coroutine (#30288)
Switched to and reset branch 'main'

Cannot create temporary file in /tmp/: No space left on device
../Python/thread.c:257:1: fatal error: error writing to /tmp/ccTo9nrX.s: No space left on device
  257 | }
      | ^
compilation terminated.
make: *** [Makefile:2392: Python/thread.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [Makefile:2392: Python/pystrtod.o] Aborted (core dumped)
../Python/pystrcmp.c:30:1: fatal error: error writing to /tmp/ccodR3vP.s: No space left on device
   30 | }
      | ^
compilation terminated.
make: *** [Makefile:2392: Python/pystrcmp.o] Error 1
../Parser/parser.c:35925:1: fatal error: error writing to /tmp/cc3DnvC8.s: No space left on device
35925 | }
      | ^
compilation terminated.
make: *** [Makefile:2392: Parser/parser.o] Error 1
../Python/symtable.c:2188:1: fatal error: error writing to /tmp/cc0vWBNO.s: No space left on device
 2188 | }
      | ^
compilation terminated.
make: *** [Makefile:2392: Python/symtable.o] Error 1
../Python/traceback.c:1334:1: fatal error: error writing to /tmp/cc6eT91n.s: No space left on device
 1334 | }
      | ^
compilation terminated.
make: *** [Makefile:2392: Python/traceback.o] Error 1
../Python/sysmodule.c:3383:1: fatal error: error writing to /tmp/ccjLnafA.s: No space left on device
 3383 | }
      | ^
compilation terminated.
make: *** [Makefile:1373: Python/sysmodule.o] Error 1
../Objects/unicodeobject.c:16167:1: fatal error: error writing to /tmp/ccmwQWIw.s: No space left on device
16167 | }
      | ^
compilation terminated.
make: *** [Makefile:2392: Objects/unicodeobject.o] Error 1

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [Makefile:2472: clean-retain-profile] Error 1 (ignored)

@kumaraditya303 kumaraditya303 deleted the issue26552 branch January 29, 2022 03:26
@kumaraditya303

kumaraditya303 commented Jan 29, 2022

Copy link
Copy Markdown
Contributor Author

@gvanrossum I think this should be backported to 3.10 and 3.9.

@gvanrossum

gvanrossum commented Jan 29, 2022

Copy link
Copy Markdown
Member

Yeah, makes sense. Adding the labels now.

@miss-islington

miss-islington commented Jan 29, 2022

Copy link
Copy Markdown
Contributor

Thanks @kumaraditya303 for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

@miss-islington

miss-islington commented Jan 29, 2022

Copy link
Copy Markdown
Contributor

Thanks @kumaraditya303 for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

@miss-islington

miss-islington commented Jan 29, 2022

Copy link
Copy Markdown
Contributor

Sorry, @kumaraditya303 and @gvanrossum, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 24cc6411adbfe5555ecd8901f1ea50caa414c908 3.9

@miss-islington

miss-islington commented Jan 29, 2022

Copy link
Copy Markdown
Contributor

Sorry @kumaraditya303 and @gvanrossum, I had trouble checking out the 3.10 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 24cc6411adbfe5555ecd8901f1ea50caa414c908 3.10

kumaraditya303 added a commit to kumaraditya303/cpython that referenced this pull request Jan 29, 2022
@kumaraditya303
bpo-26552: Fixed case where failing asyncio.ensure_future did not c… …
e514f9f 
…lose the coroutine (python#30288)
@kumaraditya303 kumaraditya303 mentioned this pull request Jan 29, 2022
Merged
@kumaraditya303

kumaraditya303 commented Jan 29, 2022

Copy link
Copy Markdown
Contributor Author

I am leaving 3.9 as it is too diverged from main.

gvanrossum pushed a commit that referenced this pull request Jan 29, 2022
@kumaraditya303
bpo-26552: Fixed case where failing asyncio.ensure_future did not c… …
a5451c9 
…lose the coroutine (#30288) (#31003)
@bedevere-bot

bedevere-bot commented Jan 29, 2022 •
edited
Loading

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE CentOS9 3.10 has failed when building commit a5451c9.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/843/builds/5) and take a look at the build logs.
  4. Check if the failure is related to this commit (a5451c9) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/843/builds/5

Summary of the results of the build (if available):

Click to see traceback logs 
remote: Enumerating objects: 13, done.        
remote: Counting objects:   8% (1/12)        
remote: Counting objects:  16% (2/12)        
remote: Counting objects:  25% (3/12)        
remote: Counting objects:  33% (4/12)        
remote: Counting objects:  41% (5/12)        
remote: Counting objects:  50% (6/12)        
remote: Counting objects:  58% (7/12)        
remote: Counting objects:  66% (8/12)        
remote: Counting objects:  75% (9/12)        
remote: Counting objects:  83% (10/12)        
remote: Counting objects:  91% (11/12)        
remote: Counting objects: 100% (12/12)        
remote: Counting objects: 100% (12/12), done.        
remote: Compressing objects:   9% (1/11)        
remote: Compressing objects:  18% (2/11)        
remote: Compressing objects:  27% (3/11)        
remote: Compressing objects:  36% (4/11)        
remote: Compressing objects:  45% (5/11)        
remote: Compressing objects:  54% (6/11)        
remote: Compressing objects:  63% (7/11)        
remote: Compressing objects:  72% (8/11)        
remote: Compressing objects:  81% (9/11)        
remote: Compressing objects:  90% (10/11)        
remote: Compressing objects: 100% (11/11)        
remote: Compressing objects: 100% (11/11), done.        
remote: Total 13 (delta 1), reused 11 (delta 1), pack-reused 1        
From https://github.com/python/cpython
 * branch                  3.10       -> FETCH_HEAD
Note: switching to 'a5451c96a14ac0c3ee7cef7b5c52ab1d783ec613'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at a5451c96a1 [bpo-26552](https://bugs.python.org/issue26552): Fixed case where failing `asyncio.ensure_future` did not close the coroutine (#30288) (#31003)
Switched to and reset branch '3.10'

../Modules/_io/textio.c:3338:1: fatal error: error writing to /tmp/cc6yGOcE.s: No space left on device
 3338 | };
      | ^
compilation terminated.
make: *** [Makefile:2085: Modules/textio.o] Error 1
make: *** Waiting for unfinished jobs....
../Programs/_testembed.c:1899:1: fatal error: error writing to /tmp/cclxhfYW.s: No space left on device
 1899 | }
      | ^
compilation terminated.
make: *** [Makefile:816: Programs/_testembed.o] Error 1
/tmp/ccqYhb34.s: Assembler messages:
/tmp/ccqYhb34.s: Fatal error: can't close Objects/unicodeobject.o: No space left on device
make: *** [Makefile:1855: Objects/unicodeobject.o] Error 1
../Modules/posixmodule.c:15828:1: fatal error: error writing to /tmp/ccmkQztW.s: No space left on device
15828 | }
      | ^
compilation terminated.
make: *** [Makefile:2046: Modules/posixmodule.o] Error 1

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [Makefile:1928: clean-retain-profile] Error 1 (ignored)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum approved these changes

@1st1 1st1 1st1 is a code owner

@asvetlov asvetlov asvetlov is a code owner

Assignees

@gvanrossum gvanrossum

Labels

topic-asyncio

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@kumaraditya303 @carlosdmz @arhadthedev @bedevere-bot @gvanrossum @miss-islington @the-knights-who-say-ni @ZeroIntensity