bpo-44348: BaseException deallocator uses trashcan by vstinner · Pull Request #28190 · python/cpython

The deallocator function of the BaseException type now uses the
trashcan mecanism to prevent stack overflow. For example, when a
RecursionError instance is raised, it can be linked to another
RecursionError through the context attribute or the traceback
attribute, and then a chain of exceptions is created. When the chain
is destroyed, nested deallocator function calls can crash with a
stack overflow if the chain is too long compared to the available
stack memory.

https://bugs.python.org/issue44348

cc @pablogsal @corona10 @nascheme

This fix for https://bugs.python.org/issue44348 doesn't try to force inlining Py_TYPE(), it doesn't change compiler options on Windows, it doesn't try to change the stack size. Instead, it uses the more portable trashcan mecanism.

I'm not sure how it is possible that test_exceptions creates a chain of exceptions long enough to crash Python. The test uses a recursion limit around 20 frames. In Visual Studio, when the stack overflow occurs, the call stack is so deep that Visual Studio says that it cannot display it fully (it's truncated).

The trashcan mechanism is known to have performance impact. Can you run some benchmarks so we can be more comfortable evaluating the solution? :)

The trashcan mechanism is known to have performance impact. Can you run some benchmarks so we can be more comfortable evaluating the solution? :)

I cannot see any overhead on my benchmark which creates up to 10 000 exceptions per iteration: https://bugs.python.org/issue44348#msg401194 Please review the benchamrk.