Multiprocessing obtaining array-CodePudding

I want to get the result_1 and result_2 arrays with the following code:

import multiprocessing as mp
import numpy as np

result_1=[]
result_2=[]

a=np.random.rand(10,10)
b=np.random.rand(7,7)

def inv_1(x):
    result_1.append(np.linalg.inv(x))

def inv_2(y):
    result_2.append(np.linalg.inv(y))


if __name__ == "__main__":

    p1 = mp.Process(target=inv_1, args=(a),)
    p2 = mp.Process(target=inv_2, args=(b),)

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    print(result_1, result_2)

However, when I run the code, I get the following output:

[] []

How can I solve this problem?

CodePudding user response：

Unlike threads, you can't share arbitrary variables between processes. To do what you're trying to do, you can create shared lists using a multiprocessing.Manager object, e.g:

import multiprocessing as mp
import numpy as np

a=np.random.rand(10,10)
b=np.random.rand(7,7)

def inv_1(x, target):
    target.append(np.linalg.inv(x))

def inv_2(y, target):
    target.append(np.linalg.inv(y))

if __name__ == "__main__":
    mgr = mp.Manager()

    result_1 = mgr.list()
    result_2 = mgr.list()

    q = mp.Queue()

    p1 = mp.Process(target=inv_1, args=(a, result_1),)
    p2 = mp.Process(target=inv_2, args=(b, result_2),)

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    print('RESULT 1:', result_1)
    print('RESULT 2:', result_2)

This does what you're trying to do, although it's not clear to me why you're doing it this way -- both result_1 and result_2 only have a single value (because you're just appending an item to an empty list), so it's not clear why you need a list in the first place.

More broadly, you might want to redesign your code so that it doesn't rely on shared variables. A common solution is to use a queue to pass data from your workers back to the main process.