Python multiprocessing not working when time.sleep is used-CodePudding

I have one function which will take different inputs and I want that function to run parallely. Below is what I tried , but it's not working due to time.sleep I think.

from multiprocessing import Process
from time import sleep
import time

def f(name):
    print('hello', name)
    time.sleep(10)

l1 = Queue()
a = Process(target=f('Tom'))
a.start()

l2 = Queue()
b = Process(target=f("Stock"))
b.start()

print (l1.get())
print (l2.get())

I want the function to run parallely. Currently the function waits for 10 seconds before it goes to the second execution.

CodePudding user response：

Use Pool and starmap:

import multiprocessing
import time

def f(name, place):
    print('hello', name, place)
    time.sleep(5)

if __name__ == '__main__':
    data = [('Louis', 'val1'), ('Paul', 'val2'), ('Alexandre', 'val3'),
            ('John', 'val4'), ('Tom', 'val5'), ('Bob', 'val6')]
    with multiprocessing.Pool(2) as pool:
        pool.starmap(f, data)

Read carefully the multiprocessing guidlines

Safe importing of main module

Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).

CodePudding user response：

See the comment posted by @juanpa.arrivillaga. Just specify as the target argument of the Process constructor the name of the function; do not call the function. The arguments to your target are specified separately as the args argument, either as a tuple or a list.

Also, since f is not returning a useful value, there is no reason to have Queue instances for the results. In any case, you are not passing the Queue instances to f and nothing is being put to the queue so I can't understand why you would be attempting to issue calls to get against these Queue instances; these calls will hang forever. The argument you do need to be passing to f is the name, as follows:

from multiprocessing import Process
#from time import sleep
import time

def f(name):
    print('hello', name)
    time.sleep(10)

start_time = time.time()

a = Process(target=f, args=('Tom',))
a.start()

b = Process(target=f, args=("Stock",))
b.start()

# Wait for processes to complete
a.join()
b.join()

elapsed_time = time.time() - start_time
print(elapsed_time)

Prints:

hello Tom
hello Stock
10.212252855300903

If you were running this on a platform such as Windows, then you would need to place the process-creating code in a special block as follows;

from multiprocessing import Process
#from time import sleep
import time

def f(name):
    print('hello', name)
    time.sleep(10)

# Required on platforms that create processes with the "spawn" method:
if __name__ == '__main__':
    start_time = time.time()

    a = Process(target=f, args=('Tom',))
    a.start()

    b = Process(target=f, args=("Stock",))
    b.start()

    # Wait for processes to complete
    a.join()
    b.join()

    elapsed_time = time.time() - start_time
    print(elapsed_time)