Home > Software engineering >  Python Multiprocessing empty array
Python Multiprocessing empty array

Time:04-21

I am just trying multiprocessing in Python and I got a problem.

from multiprocessing import Process

w = 4;
arr = []

def func(num):
    for i in range(num,50,w):
        arr.append(i)

if __name__ == '__main__':
    p1 = Process(target=func, args=(1,))
    p1.start()
    p2 = Process(target=func, args=(2,))
    p2.start()
    p1.join()
    p2.join()

After running the code I get empty values for 'arr' array.

UPDATE: Anyone who just want to figure a problem like this better use threading.

import threading

w = 4;
arr = []

def func(num):
    for i in range(num,50,w):
        arr.append(i)

if __name__ == '__main__':
    jobs = []
    jobs.append(threading.Thread(target=func, args=(1,)))
    jobs.append(threading.Thread(target=func, args=(2,)))
    jobs.append(threading.Thread(target=func, args=(3,)))
    for j in jobs:
        j.start()
    for j in jobs:
        j.join()

CodePudding user response:

Multiprocessing uses child processes, rather than threads.

You pickled the empty array arr, shipped it to 1st child, and discarded it. Then you did same with 2nd child process.

You will need to do some IPC inter process communication to synchronize children with parent. But this seems like an XY question, and your true concern is more complex than what you described. (We do appreciate that you went to the trouble of offering an MRE!)

tl;dr: No, this approach won't work, sub-processes are not threads.


Spoiler alert: the simplest and most powerful way of using multiprocessing tends to look like this:

    for result in pool( ... ):
        ...
  • Related