Home > Enterprise >  How to make multiprocessing work on Python
How to make multiprocessing work on Python

Time:03-03

I'm making a little program in Python using tkinter to make multiple objects move.

First I tried Threading to create 2 objects and make them move at the same time. So I copy/paste the function that create and move 1 object and then execute the two functions in 2 threads.

Now I wanted to try to choose the number of object moving so I got 1 function and I want to execute it multiple times at the same time. Threading was no longer a possibility.

So I tried multiprocessing and I got a problem : it does nothing... I recreated the problem with a simple code :

import multiprocessing

def pr(number):
    print(number)

p = multiprocessing.Process(target=pr,args=(40,))
p.start()

It returns an absolute nothing, nothing is printed. I think multiprocessing is not the adequate tool to do what I want but I don't understand the problem of this code.

PS : I'm using IDLE Python 3.7.2 on Windows

CodePudding user response:

This post is essentially a duplicate of Can multiprocessing Process class be run from IDLE. I found it by searching SO for "[python-idle] multiprocessing". I give a revised and expanded version of my previous answer here.

  1. As @Booboo said, an if __name __ == '__main__': block is needed, at least on Windows and perhaps macOS. With that present, multiprocessing itself runs fine when started from IDLE on Windows.

  2. The failure is with printing. When multiprocessing starts a new python process, it copies the current process sys.__stdout__ to both sys.__stdout__ and sys.stdout in the new process. By default, print writes the requested text to sys.stdout.

When a GUI program is started from an icon or somehow other than a terminal, sys.__stdout__ and sys.stdout are initialized to None, at least on Windows. IDLE replaces sys.stdout with an object whose write method sends text to IDLE's Shell. It would be useless to copy the IDLE specific sys.stdout as it would not work.

If, instead you start IDLE from a command line with, on Windows, py -m idlelib, the sys attributes are instead initialized to a normal python io object that sends text to the console/terminal. IDLE only replaces sys.stdout, so multiprocessing can copy the working sys.__stdout__ io object to both sys attributes of the new process. By default, 'print' will then print to the console, as in this example.

import multiprocessing, sys

def pr():
    print(sys.stdout)

if __name__ == '__main__':
    p = multiprocessing.Process(target=pr)
    p.start()
    p.join()

# Writes the following to the 
<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>

Printing to CommandPrompt is obviously not great, but it is the best you can do with non-IDLE code. In your particular case, of wanting to animate tkinter objects (presumably on a tkinter canvas), you should use tk/tkinter after loops instead of multiprocessing. IDLE is designed for developing tkinter programs. Searching SO for [tkinter] animate gives a few hundred hits.

CodePudding user response:

My comment about not being able to run multiprocessing under IDLE was clearly wrong as pointed out by Terry Jan Reedy.

The problem with IDLE is the same problem one has in trying to do multiprocessing from an environment such as Jupyter Notebook. That is, it can be done except that your worker function, pr in your case, must be in a separate module that is imported. If you do not do this, then you get the following exception:

AttributeError: Can't get attribute 'pr' on <module '__main__' (built-in)>

Unfortunately, by starting IDLE from the Windows start menu, that exception message was never seen. Had IDLE been started from a command prompt instead with ...

python -m idlelib

... then the AttributeError exception would have been displayed on your command prompt console.

So:

  1. Place your pr function in a file, for example pr.py, in some directory.
  2. Open up a command prompt and change the current directory to the directory that contains pr.py.
  3. Start IDLE with python -m idlelib.

Then your program becomes:

import multiprocessing
from pr import pr


if __name__ == '__main__': # required for Windows
    p = multiprocessing.Process(target=pr,args=(40,))
    p.start()
    p.join() # Wait for the completion explicitly
  • Related