bpo-44348: BaseException deallocator uses trashcan #28190
Conversation
The deallocator function of the BaseException type now uses the trashcan mecanism to prevent stack overflow. For example, when a RecursionError instance is raised, it can be linked to another RecursionError through the __context__ attribute or the __traceback__ attribute, and then a chain of exceptions is created. When the chain is destroyed, nested deallocator function calls can crash with a stack overflow if the chain is too long compared to the available stack memory.
|
cc @pablogsal @corona10 @nascheme This fix for https://bugs.python.org/issue44348 doesn't try to force inlining Py_TYPE(), it doesn't change compiler options on Windows, it doesn't try to change the stack size. Instead, it uses the more portable trashcan mecanism. I'm not sure how it is possible that test_exceptions creates a chain of exceptions long enough to crash Python. The test uses a recursion limit around 20 frames. In Visual Studio, when the stack overflow occurs, the call stack is so deep that Visual Studio says that it cannot display it fully (it's truncated). |
|
The trashcan mechanism is known to have performance impact. Can you run some benchmarks so we can be more comfortable evaluating the solution? :) |
I cannot see any overhead on my benchmark which creates up to 10 000 exceptions per iteration: https://bugs.python.org/issue44348#msg401194 Please review the benchamrk. |
LGTM, it's quite amazing that does not give notable overhead.
vstinner
merged commit fb30509
into
python:main
![@github-actions[bot]](https://web.archive.org/web/20210910033958im_/https://avatars.githubusercontent.com/in/15368?v=4){kind=small}
The deallocator function of the BaseException type now uses the
trashcan mecanism to prevent stack overflow. For example, when a
RecursionError instance is raised, it can be linked to another
RecursionError through the context attribute or the traceback
attribute, and then a chain of exceptions is created. When the chain
is destroyed, nested deallocator function calls can crash with a
stack overflow if the chain is too long compared to the available
stack memory.
https://bugs.python.org/issue44348
The text was updated successfully, but these errors were encountered: