I am hoping for some clarification on overloading Numpy universal functions in class methods.
To illustrate, here is a class myreal
with an overloaded cos
method. This overloaded method calls cos
imported from the math module.
from math import cos
class myreal:
def __init__(self,x):
self.x = x
def cos(self):
return self.__class__(cos(self.x))
def __str__(self):
return self.x.__str__()
x = myreal(3.14)
y = myreal.cos(x)
print(x,y)
This works as expected and results in values
3.14 -0.9999987317275395
And, as expected, simply trying
z = cos(x)
results in an error TypeError: must be real number, not myreal
, since outside of the myreal
class, cos
expects a float argument.
But surprisingly (and here is my question), if I now import cos
from numpy, I can call cos(x)
as a function, rather than as a method of the myreal
class. In other words, this now works:
from numpy import cos
z = cos(x)
So it seems that the myreal.cos()
method is now able to overload the Numpy global function cos(x)
. Is this "multipledispatch" behavior included by design?
Checking the type of the Numpy cos(x)
reveals that it is of type `numpy.ufunc', which suggests an explanation involving Numpy universal functions.
Any clarification about what is going on here would be very interesting and helpful.
CodePudding user response:
np.cos
given a numeric dtype array (or anything that becomes that):
In [246]: np.cos(np.array([1,2,np.pi]))
Out[246]: array([ 0.54030231, -0.41614684, -1. ])
But if I give it an object dtype array, I get an error:
In [248]: np.cos(np.array([1,2,np.pi,None]))
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
AttributeError: 'int' object has no attribute 'cos'
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
Input In [248], in <cell line: 1>()
----> 1 np.cos(np.array([1,2,np.pi,None]))
TypeError: loop of ufunc does not support argument 0 of type int which has no callable cos method
With the None
element the array is object
dtype:
In [249]: np.array([1,2,np.pi,None])
Out[249]: array([1, 2, 3.141592653589793, None], dtype=object)
ufunc
when given an object dtype array, iterates through the array and tries to use each element's "method". For something like np.add
, it looks for the .__add__
method. For functions like cos
(and exp
and sqrt
) it looks for a method of the same name. Usually this fails because most objects don't have a cos
method. In your case, it didn't fail - because you defined a cos
method.
Try np.sin
to see the error.
I wouldn't call this overloading
. It's just a quirk of how numpy
handles object dtype arrays.
CodePudding user response:
myreal.cos
is not able to overload the numpy cos
. You are calling the numpy
cos
all the time, except inside myreal
or when you write myreal.cos
. It seems that numpy functions are friendly, and when numpy cos
receives an object it doesn't know, instead of an error message, it tries to call cos
on the object.
from numpy import cos
class myreal:
def __init__(self, x):
self.x = x
def cos(self, myarg=None):
print(f'myreal-cos myarg: {myarg}')
return f'myreal-cos-{self.x}'
def __str__(self):
return self.x.__str__()
Now when we run cos, it's numpy cos, not ours:
cos(5)
0.28366218546322625
Let's try to use myarg
:
cos(5, myarg='a')
TypeError: cos() got an unexpected keyword argument 'myarg'
It's still numpy cos
, so it doesn't know the myarg keyword.
cos(myreal(2))
myreal-cos args: None
'myreal-cos-2'
This works, because numpy cos
calls our myreal.cos
. But it doesn't work with the myarg keyword.
If we try sin
:
sin(myreal(2))
TypeError: loop of ufunc does not support argument 0 of type myreal which has no callable sin method
This error message seems to make sense. numpy sin
tries to call myreal sin
, but it doesn't exist.
This friendly feature is useful when we create an array:
A = np.array([myreal(i) for i in range(4)])
array([[<__main__.myreal object at 0x7fc8c9429cf0>,
<__main__.myreal object at 0x7fc8c939b970>],
[<__main__.myreal object at 0x7fc8c939b0a0>,
<__main__.myreal object at 0x7fc8c939a8c0>]], dtype=object)
Let's call our myreal.cos
function on that array:
myreal.cos(A)
----> 9 return f'myreal-cos-{self.x}'
AttributeError: 'numpy.ndarray' object has no attribute 'x'
It doesn't work, because we didn't prepare our function to work on arrays. But numpy.cos
can, and for each element in the array it will call our myreal.cos
:
cos(A)
myreal-cos myarg: None
myreal-cos myarg: None
myreal-cos myarg: None
myreal-cos myarg: None
array(['myreal-cos-0', 'myreal-cos-1', 'myreal-cos-2', 'myreal-cos-3'],
dtype=object)
CodePudding user response:
The answers above both clarified my misunderstanding. Basically, a quirk in Numpy is what gives the impression that myreal
class methods cos
, sin
etc are "overloading" the Numpy functions cos
, sin
and so on.
Rather than exploiting this quirk in Numpy, however, it seems the more correct way to overload the cos
function so it accepts a myreal
is to use the singledispatch mechanism.
My original class definitions remain unchanged, but a "cos" function is added to a dispatch registery that is searched when looking for the appropriate function/argument to call.
from functools import singledispatch
class myreal:
def __init__(self,x):
self.x = x
def cos(self):
return myreal(cos(self.x))
def __str__(self):
return self.x.__str__()
@singledispatch
def cos(x):
import math
return math.cos(x)
cos.register(myreal,myreal.cos);
print(cos.registry.keys())
x = myreal(3.14)
y = cos(x)
print(y)
The output is :
dict_keys([<class 'object'>, <class '__main__.myreal'>])
-0.9999987317275395
This also removes a dependence on Numpy (it if it not needed).
Here is a version of the above that uses Numpy arrays (inspired by @AndrzejO, above).
from functools import singledispatch
from numpy import array
class myreal:
def __init__(self,x):
self.x = x
def cos(self):
return self.__class__(cos(self.x))
def __repr__(self):
return self.__str__()
def __str__(self):
return self.x.__str__()
@singledispatch
def cos(x):
import numpy
return numpy.cos(x)
cos.register(myreal,myreal.cos);
x = array([myreal(i) for i in range(5)])
y = cos(x)
print(y)
The output is :
[1.0 0.5403023058681398 -0.4161468365471424 -0.9899924966004454
-0.6536436208636119]
Any comments would be appreciated.