The Wayback Machine - https://web.archive.org/web/20250430205259/https://github.com/scikit-learn/scikit-learn/pull/24438
Skip to content

CI Make pytest err on PytestUnraisableExceptionWarning #24438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

jjerphan
Copy link
Member

Reference Issues/PRs

Relates to discussions in #13649, see #13649 (comment).

What does this implement/fix? Explain your changes.

If Cython directives are activated, IndexErrors (which aren't fatal) are raised and captured by pytest, but not raised again. pytest in return raises PytestUnraisableExceptionWarnings, which we must treat as errors on the CI.

@jeremiedbb
Copy link
Member

jeremiedbb commented Sep 14, 2022

I ran it locally and got 180+ failures 😄. Most of them come from the issue detected in #13649 though. You'll need to backport here the fix you found.

@jeremiedbb
Copy link
Member

looks like it did not trigger
python -m pytest --showlocals --durations=20 --junitxml=test-data.xml '--cov-config=D:\a\1\s/.coveragerc' --cov sklearn --cov-report= -Werror::DeprecationWarning -Werror::FutureWarning -Werror::numpy.VisibleDeprecationWarning -Wignore:tostring:DeprecationWarning '-Wignore:The distutils:DeprecationWarning' '-Wignore:distutils Version classes are deprecated:DeprecationWarning' -n2 --pyargs sklearn

@jjerphan
Copy link
Member Author

I do not have issues locally. The problem was introduced in #13649, but likely was not present in another place in the code-base before.

@jeremiedbb
Copy link
Member

Right, most of the failures comes from #13649. But I do have errors locally on main, from sklearn\ensemble\_hist_gradient_boosting\splitting.pyx. And as I mentionned, looking at the pytest invocation command, it seems that the warning has not be turned into an error here

@jjerphan
Copy link
Member Author

Which failure do you have exactly so that we can address them?

@jeremiedbb
Copy link
Member

jeremiedbb commented Sep 15, 2022

Imo we first need to make sure we catch them in the CI. Looking at the pytest invoke command #24438 (comment), it seems that the switching into error did not trigger

EDIT: my bad I was looking at the windows job while it's enabled in a linux job.
So it seems that we don't have these errors on linux. Let's try to enable boundscheck in a windows job

Move the handling under the "$CHECK_WARNINGS" checks.

Co-authored-by: Jérémie du Boisberranger <[email protected]>
Copy link
Member

@ogrisel ogrisel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@jeremiedbb
Copy link
Member

as discussed irl, let's try to enable boundscheck in the py38_conda_forge_mkl windows job and see if it's not too long. It means setting SKLEARN_ENABLE_DEBUG_CYTHON_DIRECTIVES : '1' for this job in azure_pipelines.yml.

@jeremiedbb
Copy link
Member

So, enabling boundschecks didn't trigger any error, but made the run 20min faster 😲

@jjerphan
Copy link
Member Author

So, enabling boundschecks didn't trigger any error, but made the run 20min faster 😲

There must be something related to shared machines and load balancing. 😄

@jjerphan
Copy link
Member Author

There is a lot of variability for CI jobs' duration; py38_conda_forge_mkl took:

I think we can note it generally but ignore this for this PR.

Copy link
Member

@jeremiedbb jeremiedbb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The timings have a lot of fluctuations but it seems that it's not longer than without enabling the bounds checks.

@jeremiedbb jeremiedbb merged commit 0ae132f into scikit-learn:main Sep 19, 2022
@jjerphan jjerphan deleted the ci/err-on-PytestUnraisableExceptionWarning branch September 19, 2022 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants