bpo-31061: fix crash in asyncio speedup module#2966
Conversation
|
this is my first cpython PR, let me know what else needs to be done to get this into any other releases it's required in. |
Sorry, something went wrong.
|
Since |
Sorry, something went wrong.
|
Oh, sorry, macro version API broke Windows build and you used public APIs to fix it. |
Sorry, something went wrong.
|
Could you add news entry? |
Sorry, something went wrong.
|
added news and reverted LRU change |
Sorry, something went wrong.
|
should we put as many fixes as we can in this PR, or create another PR with as many as we can? I need the defaultdict one too...and maybe even others as who knows what third party libraries do. I feel like we've opened a pandoras box of crashers :) |
Sorry, something went wrong.
|
I already created #2974. |
Sorry, something went wrong.
|
The latest revision LGTM. But can you use |
Sorry, something went wrong.
|
@1st1 should I remove the one in NEWS? Also was hoping I could add a testcase for Task but couldn't figure out how to repro it, especially given Task inherits from Future so I don't want to trigger the Future's crash |
Sorry, something went wrong.
Yes, please!
I'm trying to come up with a test. |
Sorry, something went wrong.
|
The following test crashes unpatched CPython: import gc
def test_task_del_collect(self):
async def coro():
return Evil()
class Evil:
def __del__(self):
gc.collect()
for i in range(100):
task = self.new_task(self.loop, coro())
self.loop.run_until_complete(task) |
Sorry, something went wrong.
|
The test also fails if we fix only Task, but not Future. |
Sorry, something went wrong.
|
so I had something similar but it doesn't crash: import gc
import asyncio
class Evil:
def __del__(self):
gc.collect()
async def run():
return Evil()
loop = asyncio.get_event_loop()
for i in range(100):
loop.run_until_complete(asyncio.Task(run()))
# f = asyncio.Future()
# f.set_result(Evil()) |
Sorry, something went wrong.
|
ah, figured it out, had to have multiple tasks pending in the run loop. |
Sorry, something went wrong.
|
ya realized as I was going to work, this should fix it |
Sorry, something went wrong.
|
this set of PRs is a massive win for Python. It would be nice if a core dev could look and find a way to nip it at the bud to avoid these hacky calls, as shown they're highly error prone. Who knows how many third party modules are missing them too. |
Sorry, something went wrong.
methane
left a comment
There was a problem hiding this comment.
LGTM
Sorry, something went wrong.
(cherry picked from commit de34cbe)
|
Do we have C Task/Future in 3.5? |
Sorry, something went wrong.
|
No. asyncio speedup is introduced at 3.6 |
Sorry, something went wrong.
https://bugs.python.org/issue31061
follows what was done in: https://bugs.python.org/issue26617
fallout in: http://bugs.python.org/issue31095