The Wayback Machine - https://web.archive.org/web/20250704112431/https://github.com/matplotlib/matplotlib/pull/25832
Skip to content

[BUG] Prevent under the hood downcasting of values #25832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 15, 2023

Conversation

abcnishant007
Copy link
Contributor

@abcnishant007 abcnishant007 commented May 8, 2023

I think this behaviour was not intended. Spotted the issue due to a question posted on Stackoverflow https://stackoverflow.com/questions/76195580/when-plotting-autocorrelation-function-plot-getting-ufunctypeerror

When using a list of integers in the xcorr function, the following lines:

if normed:
correls /= np.sqrt(np.dot(x, x) * np.dot(y, y))

result in an under-the-hood downcasting to int, and as a result, the following error is thrown:

  File "---/pythonProject/lib/python3.8/site-packages/matplotlib/axes/_axes.py", line 2120, in xcorr
    correls /= np.sqrt(np.dot(x, x) * np.dot(y, y))
TypeError: ufunc 'true_divide' output (typecode 'd') could not be coerced to provided output parameter (typecode 'l') 
according to the casting rule ''same_kind''

The obvious fix is to remove in-place update and use standard assignment.

PR summary

PR checklist

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for opening your first PR into Matplotlib!

If you have not heard from us in a while, please feel free to ping @matplotlib/developers or anyone who has commented on the PR. Most of our reviewers are volunteers and sometimes things fall through the cracks.

You can also join us on gitter for real-time discussion.

For details on testing, writing docs, and our review process, please see the developer guide

We strive to be a welcoming and open project. Please follow our Code of Conduct.

@abcnishant007
Copy link
Contributor Author

Reopened to trigger waiting tests.

@tacaswell
Copy link
Member

Thank you for working on this.

I would describe this is more forcing an up-conversion that preventing a down conversion (as it looks like np.corrolate is preserving the types passed in) but that is neither here nor there. The current best work-around for users is to do np.asarray(data, dtype=float) before passing it in to ensure that we are working with floats all the way through.

Could you add a simple test that passing in a list of ints does not raise?

@tacaswell tacaswell added this to the v3.7.2 milestone May 8, 2023
@abcnishant007
Copy link
Contributor Author

Sure thing, I will add the test case today.

Thanks for the review!

@abcnishant007
Copy link
Contributor Author

@tacaswel I wrote the test, but in one of the CI tests, it compares against the original main and fails. Am I missing something trivial here? For your reference, the permalink of the error is reproduced below:

https://dev.azure.com/matplotlib/matplotlib/_build/results?buildId=33482&view=logs&j=aa39683c-d595-5f65-0b25-75adb2a2e3f3&t=2551f7a6-26d7-58a2-495a-78f1a540557c&l=292

@tacaswell
Copy link
Member

        if normed:
>           correls = correls / np.sqrt(np.dot(x, x) * np.dot(y, y))
E           RuntimeWarning: overflow encountered in scalar multiply

I am very confused why this is only happening on windows, but this looks like a real warning from numpy and our tests are set to fail on any warning.

Given that we are looking for "does this fail", what is the simplest (smallest) input data we can get away with using?

@ksunden
Copy link
Member

ksunden commented May 9, 2023

Alternative would be to upcast to float sooner which would likely elide the precision warning (though may still have slight imprecision due to inexact float calcs).

e.g.

    correls = np.correlate(x, y, mode="full").astype(float)

(And then keep the /=)

@abcnishant007
Copy link
Contributor Author

abcnishant007 commented May 10, 2023

        if normed:
>           correls = correls / np.sqrt(np.dot(x, x) * np.dot(y, y))
E           RuntimeWarning: overflow encountered in scalar multiply

I am very confused why this is only happening on windows, but this looks like a real warning from numpy and our tests are set to fail on any warning.

Given that we are looking for "does this fail", what is the simplest (smallest) input data we can get away with using?

This may be due to the system default long in Windows vs Linux. So, trying explicit int64 now.

Copy link
Member

@tacaswell tacaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modulo squashing or squash-merge.

@tacaswell
Copy link
Member

Looks good @abcnishant007 !

Are you comfortable squashing your commits (see https://matplotlib.org/stable/devel/development_workflow.html#rewriting-commit-history)?

@abcnishant007 abcnishant007 changed the title Prevent under the hood downcasting of values [BUG] Prevent under the hood downcasting of values May 12, 2023
@abcnishant007
Copy link
Contributor Author

Thanks for the review @tacaswell. I targeted two commits- 1 for the change and 1 for the test. I did the cleaning and squashing using rebase, but while changing the commit message of the first commit, there was a dummy merge. Now, GitHub won't let me squash that merge commit anymore.

If this is unacceptable, I can create a new pull request with exactly two commits and the same changes as this one. What do you think?

@abcnishant007 abcnishant007 requested a review from tacaswell May 12, 2023 14:16
@tacaswell
Copy link
Member

tacaswell commented May 12, 2023

It looks like you fell into the trap of: https://matplotlib.org/devdocs/devel/development_workflow.html#pushing-with-force

You are probably [these are destructive commands, understand them before you run them!]

git reset --hard SHA_OF_COMMIT_YOU WANT
git push --force-with-lease

away from getting rid of the merge (and duplicate) commits.

@abcnishant007
Copy link
Contributor Author

Thanks for the guidance! Yes, that was the trap. Finally, I managed to fix it by:

  • using reset --hard
  • and then re-introducing the two commits.
  • finally push with --force

Now, we have a clean commit history of two commits, one for the fix, other for the test.

@tacaswell tacaswell closed this May 12, 2023
@tacaswell tacaswell reopened this May 12, 2023
@tacaswell
Copy link
Member

tried to re-trigger CI to pick up a fix on main for the mypy error.

@abcnishant007
Copy link
Contributor Author

I think the mypy subtest issue is resolved after re-running the tests.

@ksunden ksunden merged commit 9e1359d into matplotlib:main May 15, 2023
meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this pull request May 15, 2023
ksunden added a commit that referenced this pull request May 15, 2023
…832-on-v3.7.x

Backport PR #25832 on branch v3.7.x ([BUG] Prevent under the hood downcasting of values)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

5 participants