How do I do multithreading correctly in PyQt5 while reading and writing to a file simultaneously?-CodePudding

In one window, I have a button that, when clicked, I want to execute a method from another module. This method takes an indeterminate amount of time to execute and depends on user input in the terminal. This method creates a file and repeatedly opens it, writes things to the file, and closes the file. At the same time this is running I have a matplotlib graph widget in the window with a plot that I want to update each time something new is written to the file by reading and plotting data from the most recent line of the file.

To check for changes to the file, I'm using QFileSystemWatcher. Right now, nothing happens while the userInputFunction() is running, but when it finishes I get "data/runName_Rec.txt dataFileCreated". If I then manually edit the file in any way the plotting happens as it should. So it seems the watcher only starts working again and sees that there has been a change in the directory after the userInputFunction() finishes.

How do I do the multithreading correctly so that the watcher works while userInputFunction() is running?

As I understand it, nothing in my application will respond until the user input function finishes if I have it running in the main thread of my QT program. To address this I tried moving the execution of the user input method into a worker thread following the example here: https://realpython.com/python-pyqt-qthread/. In this worker I have two methods. One simply does a for loop with a sleep() that takes a while exactly like the example. The other runs my userInputFunction(). The for loop method, run(), does not freeze the GUI. However, runScan() which exicutes the actual process I want does still freeze the GUI. I'm not sure what is going on here. I'm not sure if this means I'm not doing the threading properly or if something else is going on.

Here is a simplified sample of the relevant parts of my code.

from PyQt5 import QtWidgets, uic, QtCore, QtGui
from pyqtgraph import PlotWidget
from PyQt5.QtCore import QObject, QThread, pyqtSignal
import pyqtgraph as pg
import sys
from PyQt5.QtWidgets import QApplication, QWidget, QInputDialog, QLineEdit, QFileDialog, QMainWindow
import os
from os.path import exists
import csv
import numpy as np
import pandas as pd

import myModule

dirname = os.path.dirname(__file__)

# Create a worker class
class Worker(QObject):
    finished = pyqtSignal()

    #This works without freezing the GUI
    def run(self):
        """Long-running task."""
        for i in range(5):
            time.sleep(1)
            print("step ",i," done!")
        self.finished.emit()

    #This freezes the GUI
    def runScan(self,param1,param2):
        """Long-running task with user input from terminal."""
        myModule.userInputFunction(param1,param2)
        self.finished.emit()


class someWindow(QtWidgets.QMainWindow):

    def __init__(self, *args, **kwargs):
        super(someWindow, self).__init__(*args, **kwargs)

        #Load the UI Page
        uic.loadUi('somewindow.ui', self)

        self.directoryPath = "data"

        self.fs_watcher = QtCore.QFileSystemWatcher()

        self.fs_watcher.addPath(self.directoryPath)
        self.fs_watcher.directoryChanged.connect(self.dataFileCreated)
        
        self.StartScanButton.clicked.connect(self.runLongTask)
        self.EndScanButton.clicked.connect(self.endScan)

    def dataFileCreated(self):
        self.filePath = os.path.join(dirname, "data/"  self.runNameBox.toPlainText() "_Rec.txt")
        print(self.filePath   "dataFileCreated")
        self.fs_watcher.addPath(self.filePath)
        self.fs_watcher.fileChanged.connect(self.update_graph)

    def update_graph(self):
        if exists(self.path):
            print("file exists!")
            #then read the filePath.txt and plots the data
    else:
        print("file doesn't exist yet")

    def endScan(self):
        #change some display things

    def runLongTask(self):
        # Create a QThread object
        self.thread = QThread()
        # Create a worker object
        self.worker = Worker()
        # Move worker to the thread
        self.worker.moveToThread(self.thread)
        # Connect signals and slots
        #This results in the GUI freezing
        self.thread.started.connect(self.worker.runScan(self.param1,self.param2))
        #This DOES NOT result in the GUI freezing
        #self.thread.started.connect(self.worker.run)

        self.worker.finished.connect(self.thread.quit)
        self.worker.finished.connect(self.worker.deleteLater)
        self.thread.finished.connect(self.thread.deleteLater)
        # Start the thread
        self.thread.start()

        # Final resets
        self.thread.finished.connect(
            lambda: print("long task done!")
        )

if __name__ == "__main__":
    app = QtWidgets.QApplication(sys.argv)
    w = someWindow()
    w.show()
    sys.exit(app.exec_())

CodePudding user response：

The GUI freezes in your case because of this line:

self.thread.started.connect(self.worker.runScan(self.param1,self.param2))

This causes runScan to be executed from the main thread and blocking anything until completed, including the connect.

It's also a serious error, as connect always expects a callable as argument, and as soon as runScan eventually completes its job it returns None and your program will crash.

Assuming that the parameters are being added when the thread is created, you could add those to the Worker constructor and then execute the required code in run:

class Worker(QObject):
    finished = pyqtSignal()
    def __init__(self, param1, param2):
        super().__init__()
        self.param1 = param1
        self.param2 = param2

    def run(self):
        myModule.userInputFunction(self.param1, self.param2)
        self.finished.emit()


class someWindow(QtWidgets.QMainWindow):
    # ...
    def runLongTask(self):
        self.thread = QThread()
        self.worker = Worker(self.param1, self.param2)
        self.worker.moveToThread(self.thread)
        self.thread.started.connect(self.worker.run)

Note that it's not like the QFileSystemWatcher "stops" while processing: the problem is that by running runScan in the main thread you completely block the main thread (not only the UI), preventing the watcher to handle the events that the OS sends it to notify about the changes.