I have three processes running concurrently. These Processes receive data at microsecond interval with the help of Queue. The processes are never joined and basically live in a while loop till program is terminated. Calculations on the received data are made and if certain conditions are met, a variable needs to be sent back to the gui. All three processes use the same functions.

class Calc:

    def __init__(self):
        self.x = [0,0,0,0,0]
        self.y = [0,0,0,0,0]

    def doCalculations(self, x, y, i):
        #do calculation stuff.
        #if certain conditions are met:
        #self.returnFunction(i)

    def returnFunction(self, i):
        return (self.x[i] - self.y[i]), i # These are two integers

def startProcesses(q: multiprocessing.Queue):
    calc = Calc()
    while True:
        x, y, i = q.get()
        calc.doCalculations(x,y,i)

def main():
    q1 = Queue()
    q2 = Queue()
    q3 = Queue()
    p1 = Process(target=startProcesses, args=(q1,))
    p2 = Process(target=startProcesses, args=(q2,))
    p3 = Process(target=startProcesses, args=(q3,))
    p1.start()
    p2.start()
    p3.start()
    #run() which is the structure that feeds the data to the queues.

if __name__ == '__main__':
    main()

Now i want to add the gui, which is fine but how do i sent the data from the process back to the gui? Do i make another Queue() in the Process itself? This would add 3 more queues because the processes dont share memory. Or is there a more elegant/simpler way to do this?

CodePudding user response：

You just need to pass one queue if all you want is the result. Create one queue and pass it to all three processes. They can then add results on the queue itself while you can receive them from the parent process.

However, if you want to identify which results came from which process, and only get those specific results, then you can use a managed dictionary (reference)

For example, create a dictionary with keys p1, p2 and p3. These will be where each process will store their results. Then pass this dictionary to each process. When the process wants to return something, make it edit the value inside the relevant key (process 1 edits p1, etc., you can pass an additional string to each process which points to the key it should edit). Since this dictionary is synchronized across processes, these values will be available to the parent process as well. This way, you would not have to create three separate structures.

To create a managed dictionary:

from multiprocessing import Manager

if __name__ == "__main__":
    manager = Manager()
    d = manager.dict({'p1': None, 'p2': None, 'p3': None})

Be sure to close the manager using manager.shutdown() so it gets garbage collected as well.