◐ Shell
reader mode source ↗
Skip to content

bpo-30856: Update TestResult early, without buffering in _Outcome#28180

Merged
serhiy-storchaka merged 3 commits into
python:mainfrom
serhiy-storchaka:unittest-result-immediate-update
Sep 19, 2021
Merged

bpo-30856: Update TestResult early, without buffering in _Outcome#28180
serhiy-storchaka merged 3 commits into
python:mainfrom
serhiy-storchaka:unittest-result-immediate-update

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Sep 5, 2021 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

TestResult methods addFailure(), addError(), addSkip() and
addSubTest() are now called immediately after raising an exception
in test or finishing a subtest. Previously they were called only
after finishing the test clean up.

https://bugs.python.org/issue30856

@serhiy-storchaka
bpo-30856: Update TestResult early, without buffering in _Outcome …
ee4b771 
TestResult methods addFailure(), addError(), addSkip() and
addSubTest() are now called immediately after raising an exception
in test or finishing a subtest.  Previously they were called only
after finishing the test clean up.
@serhiy-storchaka serhiy-storchaka added the type-feature A feature request or enhancement label Sep 5, 2021
@vstinner vstinner mentioned this pull request Sep 6, 2021
Closed
@serhiy-storchaka @ambv
Update Lib/unittest/case.py …
f2d5ffc 
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
@ambv

ambv commented Sep 17, 2021

Copy link
Copy Markdown
Contributor

BTW, the original introduction of _Outcome was in BPO-10611 and I checked that this PR (GH-28180) preserves the fixed behavior that Michael wanted in that issue. 👍🏻

@serhiy-storchaka

serhiy-storchaka commented Sep 19, 2021

Copy link
Copy Markdown
Member Author

I am merging it now. We can continue the discussion if you show a case in which additional checks would be helpful.

@serhiy-storchaka serhiy-storchaka merged commit 664448d into python:main Sep 19, 2021
@serhiy-storchaka serhiy-storchaka deleted the unittest-result-immediate-update branch September 19, 2021 12:24
niyas-sait pushed a commit to niyas-sait/cpython that referenced this pull request Sep 21, 2021
@serhiy-storchaka @niyas-sait
bpo-30856: Update TestResult early, without buffering in _Outcome (py… …
15442f8 
…thonGH-28180)

TestResult methods addFailure(), addError(), addSkip() and
addSubTest() are now called immediately after raising an exception
in test or finishing a subtest.  Previously they were called only
after finishing the test clean up.
yilei added a commit to yilei/abseil-py that referenced this pull request Jul 18, 2022
@yilei @copybara-github
Make TempFileCleanup.SUCCESS work with Python 3.11 after python/cpy… …
f2a901e 
…thon#28180.

After the change, the errors from the test case were no longer buffered in `_Outcome`. Instead, we need to take a note of errors on the `TestResult` before, then compare errors after.

PiperOrigin-RevId: 461070558
Change-Id: I7d9f769cd46143a886bc5edeb224804acd8dbaf9
@jiasli jiasli mentioned this pull request Oct 19, 2023
Open
@jiasli

jiasli commented Oct 19, 2023

Copy link
Copy Markdown
Contributor

This change makes it no longer possible to get test errors during the execution of a cleanup function added by unittest.case.TestCase.addCleanup.

The errors are now immediately sent to result by calling addError on the result object:

cpython/Lib/unittest/case.py

Line 74 in 26a0282

_addError(self.result, test_case, exc_info)

cpython/Lib/unittest/case.py

Lines 94 to 99 in 26a0282

def _addError(result, test, exc_info):
if result is not None and exc_info is not None:
if issubclass(exc_info[0], test.failureException):
result.addFailure(test, exc_info)
else:
result.addError(test, exc_info)

The actually implementation of result may not necessarily be a subclass unittest.result.TestResult. For example, pytest passes _pytest.unittest.TestCaseFunction as result. There is no consistent way of retrieving test errors from result, as the implementation of result is test-runner-dependent.

So the question is: After this change, how can a cleanup function get test errors and act accordingly?

Thanks for your help in advance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ambv ambv ambv left review comments

@voidspace voidspace Awaiting requested review from voidspace

Assignees

No one assigned

Labels

type-feature A feature request or enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@serhiy-storchaka @ambv @jiasli @the-knights-who-say-ni @bedevere-bot