My method compare_values() takes a list of values as input. For each value in the list, the method performs some long and slow calculations on the value, and then compares the value to a the variable self.lowest_value. If the calculated value is smaller than self.lowest_value, then self.lowest_value is assigned the calculated value.

My initial problem was that the method was low due to the long calculations. For a list with 10 elements it took my method 11.5 second to run. I've never used multiprocessing before, but I thought this would be a good opportunity to learn.

Before using multiprocessing I had defined self.lowest_value = 999, but self.lowest_value wouldn't get updated as I ran my method. I now understand this is because the instance variable wasn't shared between the processes. So now I'm using multiprocessing.Manager() to define self.lowest_value = multiprocessing.Manager().list([999]) so that I've got a shared variable.

But now the problem is: I get TypeError: '<' not supported between instances of 'int' and 'ListProxy'.

I'd appreciate some help! Thanks!

Packages

import time
import multiprocessing

MWE Method

class Calculations():
    def __init__(self):
        self.lowest_value = multiprocessing.Manager().list([999])

    def compare_values(self, candidate_values):
        for candidate_value in candidate_values:
            time.sleep(1) # REPRESENTS A SLOW CAULCULATION MODIFYING candidate_value
            if candidate_value < self.lowest_value:
                self.lowest_value[0] = candidate_value

With multiprocessing

if __name__  ==  '__main__':

    start_time = time.perf_counter()
    my_calculation = Calculations()
    my_values = [0,-2,6,-7, 3, -3, 0, 3, 2, 7]
    my_values1 = my_values[:5]
    my_values2 = my_values[5:]
    p1 = multiprocessing.Process(target=my_calculation.compare_values, args = (my_values1,)) 
    p2 = multiprocessing.Process(target=my_calculation.compare_values, args = (my_values2,)) 
    p1.start()
    p2.start()
    p1.join()
    p2.join()
    end_time = time.perf_counter()
    print(f'It took {end_time-start_time: .2f} second(s) to complete.')
    print(f'The lowest value is {my_calculation.lowest_value}.')   

ERROR

 TypeError: '<' not supported between instances of 'int' and 'ListProxy'

CodePudding user response：

The problem is that you are comparing candidate_value and self.lowest_value. self.lowest_value is a list in your shared memory. Since your shared memory is only an int I strongly suggest to use manager.Value instead, like this:

self.lowest_value = multiprocessing.Manager.Value('i', 999)

Next, to acces your shared Value, you need to access its object value, or you will be accessing the shared memory object and not its value (explaining your error):

if candidate_value < self.lowest_value.value:

To conclude, your program should look like this:

import time
import multiprocessing

class Calculations():   
    def __init__(self):
        self.lowest_value = multiprocessing.Manager().Value('i',999)

    def compare_values(self, candidate_values):
        for candidate_value in candidate_values:
            time.sleep(1) # REPRESENTS A SLOW CAULCULATION MODIFYING candidate_value
            if candidate_value < self.lowest_value.value:
                self.lowest_value.value = candidate_value
                

if __name__  ==  '__main__':
    
    start_time = time.perf_counter()
    my_calculation = Calculations()
    my_values = [0,-2,6,-7, 3, -3, 0, 3, 2, 7]
    my_values1 = my_values[:5]
    my_values2 = my_values[5:]
    p1 = multiprocessing.Process(target=my_calculation.compare_values, args = (my_values1,)) 
    p2 = multiprocessing.Process(target=my_calculation.compare_values, args = (my_values2,)) 
    p1.start()
    p2.start()
    p1.join()
    p2.join()
    end_time = time.perf_counter()
    print(f'It took {end_time-start_time: .2f} second(s) to complete.')
    print(f'The lowest value is {my_calculation.lowest_value.value}.')