Improving the error message when adding i8 array to u8 array #14828

gregreen · 2019-11-04T17:16:41Z

When attempting to add an ndarray of type int64 to an ndarray of type uint64, numpy throws a TypeError:

TypeError: Cannot cast ufunc add output from dtype('float64') to dtype('uint64') with casting rule 'same_kind'

The error message incorrectly identifies the type of the first array as float64, when it is actually int64.

Reproducing code example:

import numpy as np
x = np.arange(5, dtype='u8')
y = np.arange(5, dtype='i8')
x += y

Error message:

TypeError: Cannot cast ufunc add output from dtype('float64') to dtype('uint64') with casting rule 'same_kind'

Numpy/Python version information:

Tested with Python 3:

1.16.4 3.6.8 (default, Oct  7 2019, 12:59:55) 
[GCC 8.3.0]

Also with Python 2:

('1.15.1', '2.7.15+ (default, Oct  7 2019, 17:39:04) \n[GCC 7.4.0]')

eric-wieser · 2019-11-04T17:41:09Z

The error message incorrectly identifies the type of the first array as float64

That would be true if the message said "Cannot cast ufunc add input", but it actually uses the word "output".

The meaning here is that the output performed by np.add(uint64, int64) is of type float64, but you've requested it be placed in a uint64.

Can you suggest a clearer error message? The code that generates it is here:

numpy/numpy/core/_exceptions.py

Lines 108 to 116 in 463acda

    
           def __str__(self): 
        
               # only show the number if more than one output exists 
        
               i_str = "{} ".format(self.out_i) if self.ufunc.nout != 1 else "" 
        
               return ( 
        
                   "Cannot cast ufunc {!r} output {}from {!r} to {!r} with casting " 
        
                   "rule {!r}" 
        
               ).format( 
        
                   self.ufunc.__name__, i_str, self.from_, self.to, self.casting 
        
               )

gregreen · 2019-11-04T22:54:58Z

Okay, I hadn't appreciated the mechanics of what's going on behind the scenes with x += y. It's surprising that adding a signed integer array to an unsigned integer array in-place involves floats during an intermediate step, which is why I found the error message confusing. I wouldn't have guessed that np.add(x, y) would produce a float64, although I can now see why one might make that choice (in my original problem, y was the output of np.count_nonzero, and it's just as surprising that that function returns a signed integer!).

For this particular problem, something like

Output of np.add(uint64, int64) is of type float64, which cannot be cast to uint64 with casting rule 'same_kind'

would be more informative, but it's a much more specific error message that wouldn't work for all ufunc functions.

eric-wieser · 2019-11-04T23:21:46Z

That seems like a pretty reasonable suggestion - note that in order to get the extra dtype information, you (or whoever wants to improve this) would need to modify these lines too:

numpy/numpy/core/src/umath/ufunc_type_resolution.c

Lines 246 to 252 in 7b8513f

    
           } else if (operands[i] != NULL) { 
        
               if (!PyArray_CanCastTypeTo(dtypes[i], 
        
                                       PyArray_DESCR(operands[i]), casting)) { 
        
                   return raise_output_casting_error( 
        
                       ufunc, casting, dtypes[i], PyArray_DESCR(operands[i]), i); 
        
               } 
        
           }

With that in mind, most of the exceptions would probably want to change in the same way - but that's fine, they're not public API thanks to @_display_as_base :)

Another possible wording:

Output of ufunc add(uint64, int64, out=uint64) resolved to the add(float64, float64, out=float64) loop, but float64 cannot be cast to uint64 with casting rule 'same_kind'

or even

Output of ufunc add(uint64, int64, out=uint64) resolved to the add(float64, float64, out=float64) loop, but np.cancast(float64, uint64, casting='same_kind') is False

where the former types are PyArray_DESCR(operands[i]) and the latter are dtypes[i].

aditisingh2362 · 2019-11-10T14:43:57Z

I want to work on this.
I'm new to open source and I was looking for some good first issue, and I found this one. Are there any prerequisites?

eric-wieser · 2019-11-10T15:46:29Z

There's already a PR for this, but it seems we forgot to link it...

tinaoberoi · 2020-04-21T01:13:36Z

Is this issue still active? If yes I would like to take this up. @eric-wieser

eric-wieser · 2020-04-21T11:37:29Z

Let's wait and see if any progress is made on #14843

eric-wieser changed the title ~~Incorrect error message when adding i8 array to u8 array~~ Improving the error message when adding i8 array to u8 array Nov 4, 2019

eric-wieser added 01 - Enhancement 04 - Documentation component: numpy.ufunc good first issue labels Nov 4, 2019

micha2718l linked a pull request that will close this issue Nov 10, 2019

ENH: DOC: improving error messages for casting errors in ufuncs #14828 #14843

Open

Jun	JUL	Aug
	17
2019	2020	2021

numpy / numpy

Improving the error message when adding i8 array to u8 array #14828

Improving the error message when adding i8 array to u8 array #14828

gregreen commented Nov 4, 2019

eric-wieser commented Nov 4, 2019

gregreen commented Nov 4, 2019 •

edited

eric-wieser commented Nov 4, 2019 •

edited

aditisingh2362 commented Nov 10, 2019

eric-wieser commented Nov 10, 2019

tinaoberoi commented Apr 21, 2020 •

edited

eric-wieser commented Apr 21, 2020

numpy / numpy

Sponsor numpy/numpy

Join GitHub today

Improving the error message when adding i8 array to u8 array #14828

Improving the error message when adding i8 array to u8 array #14828

Comments

gregreen commented Nov 4, 2019

Reproducing code example:

Error message:

Numpy/Python version information:

eric-wieser commented Nov 4, 2019

gregreen commented Nov 4, 2019 • edited

eric-wieser commented Nov 4, 2019 • edited

aditisingh2362 commented Nov 10, 2019

eric-wieser commented Nov 10, 2019

tinaoberoi commented Apr 21, 2020 • edited

eric-wieser commented Apr 21, 2020

gregreen commented Nov 4, 2019 •

edited

eric-wieser commented Nov 4, 2019 •

edited

tinaoberoi commented Apr 21, 2020 •

edited