Python Multiprocessing manager prunes some methods-CodePudding

I've found that Manager rarely works in practice with most of the custom objects as it removes some class members. See an example:

from multiprocessing import Manager
from scipy.interpolate import CubicSpline
cs_pure = CubicSpline([1,2,3], [1,2,3])
test_pure = cs_pure(1)
m = Manager()
m.register('CubicSpline', CubicSpline)
cs_m = m.CubicSpline([1,2,3], [1,2,3])
test_m = cs_m(1)

This will raise an exception:

TypeError: 'AutoProxy[CubicSpline]' object is not callable

Surprisingly, most of other methods (such as integrate, differentiate) work as expected. The __call__ method is non-existent, though.

Is there are a workaround to this?

CodePudding user response：

Did you mean test_m = cs_m(1)?

First of all from my experience you have to register a managed class with the manager's class before the manager server is started:

from multiprocessing.managers import SyncManager
from multiprocessing import Manager
from scipy.interpolate import CubicSpline

SyncManager.register('CubicSpline', CubicSpline)
m = Manager()
...

So I am not sure how you got as far as you did. It also more usual to create your own manager class if you do not need all the other registered managed classes that are implemented with the SyncManager (e.g. dict). I will demonstrate that in the following code (use this technique or not as you wish).

I believe the problem is that the __call__ method is not automatically being generated in the proxy class that is generated for you when you do not define your own proxy class. Rather than defining your own proxy class, I think the simpler solution is to subclass the CubicSpline class and add an additional method such as evaluate_polynomial that will simply call the superclass __call__ method:

from multiprocessing.managers import BaseManager
from scipy.interpolate import CubicSpline

class CubicSplineManager(BaseManager):
    pass

class MyCubicSpline(CubicSpline):
    def evaluate_polynomial(self, *args, **kwargs):
        return self.__call__(*args, **kwargs)

if __name__ == '__main__':
    cs_pure = CubicSpline([1,2,3], [1,2,3])
    test_pure = cs_pure(1)
    print(test_pure)
    CubicSplineManager.register('CubicSpline', MyCubicSpline)
    m = CubicSplineManager()
    m.start() # We must explicitly start the server
    cs_m = m.CubicSpline([1,2,3], [1,2,3])
    test_m = cs_m.evaluate_polynomial(1)
    print(test_m)

Prints:

1.0
1.0