I'm trying to run the following code, it's main aim is to do much more complex procedures , I simplified it with two loops.

I keep getting the errors below: (I searched the error with no luck)

from multiprocessing import Process

def load():
    for i in range(0, 10000):
        print("loadddinngg", i)

def copy(p1):
    # fetch all files
    while p1.is_alive():
        for i in range(0, 100000):
            print("coppppyyy", i)

class multithreading:
    def __init__(self):
        p1 = Process(target=load, args=())

        p2 = Process(target=copy, args=( p1,))
        p1.start()
        p2.start()
        p1.join()
        p2.join()

 File "C:\Users\untitled10\toDelete.py", line 19, in __init__
    p2.start()
  File "C:\Program Files\Python38_64bit\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Program Files\Python38_64bit\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Program Files\Python38_64bit\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Program Files\Python38_64bit\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Program Files\Python38_64bit\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'weakref' object

CodePudding user response：

The args to a Process object are pickled in order to send that information to the new process (so that it can unpickle and recreate the same Python objects in its own address space). Process objects cannot themselves be pickled.

Separately, as stated in the documentation:

Note that the start(), join(), is_alive(), terminate() and exitcode methods should only be called by the process that created the process object.

We therefore cannot expect to use is_alive within the spawned processes.

To communicate between processes, use multiprocessing.Queue. For this specific case, we want p2 to stop when p1 is done. That means, p1 has to tell p2 that it is just about to finish, and then p2 receives that message and responds by stopping.

A trivial example might look like:

from multiprocessing import Process, Queue
from queue import Empty # exception raised when the queue is empty
import sys

def load(q):
    do_some_work()
    # the actual object that we `put` in the Queue
    # doesn't matter for this trivial example.
    q.put(None) 

def copy(q):
    while True:
        do_some_other_work()
        try:
            q.get()
        except Empty:
            pass # the message wasn't sent yet.
        else:
            return # it was sent, so stop this thread too.

if __name__ == '__main__':
    q = Queue()
    p1 = Process(target=load, args=(q,))
    p2 = Process(target=copy, args=(q,))
    p1.start()
    p2.start()
    p1.join()
    p2.join()