I am working on my first application. In one window, I have a button that, when clicked, I want to execute a method from another module. This method takes an indeterminate amount of time to execute and depends on user input in the terminal. This method creates a file and repeatedly opens it, writes things to the file, and closes the file. At the same time this is running I have a matplotlib graph widget in the window with a plot that I want to update each time something new is written to the file by reading and plotting data from the most recent line of the file.

As I understand it, nothing in my application will respond until the user input function finishes if I have it running in the main thread of my QT program. To address this I tried moving the execution of the user input method into a worker thread. In the way I have done this I'm not convinced it is working. As a test I tried making a QTimer that tried to read the file and plot it every second (with some added stuff to check if the file exists). This prints that the file doesn't exist yet right until the long task, and then does nothing, until the long task is done and then starts reading and plotting the file every second. I'm not sure if this means I'm not doing the threading properly or if something else is going on.

To check for changes to the file, I've tried using QFileSystemWatcher. UPDATE: Right now, nothing happens while the userInputFunction() is running, but when it finishes I get "data/runName_Rec.txt dataFileCreated". If I then manually edit the file in any way the plotting happens as it should. But I still want to thread it correctly so that the watcher works while I'm running userInputFunction()

Here is a simplified sample of the relevant parts of my code. Sorry for any bad style issues.

from PyQt5 import QtWidgets, uic, QtCore, QtGui
from pyqtgraph import PlotWidget
from PyQt5.QtCore import QObject, QThread, pyqtSignal
import pyqtgraph as pg
import sys
from PyQt5.QtWidgets import QApplication, QWidget, QInputDialog, QLineEdit, QFileDialog, QMainWindow
import os
from os.path import exists
import csv
import numpy as np
import pandas as pd

import myModule

dirname = os.path.dirname(__file__)
class Worker(QObject):
    finished = pyqtSignal()

    def run(self,param1,param2):
        """Long-running task with user input from terminal."""
        myModule.userInputFunction(param1,param2)
        self.finished.emit()

class someWindow(QtWidgets.QMainWindow):

    def __init__(self, *args, **kwargs):
        super(someWindow, self).__init__(*args, **kwargs)

        #Load the UI Page
        uic.loadUi('somewindow.ui', self)

        self.directoryPath = "data"

        self.fs_watcher = QtCore.QFileSystemWatcher()

        self.fs_watcher.addPath(self.directoryPath)
        self.fs_watcher.directoryChanged.connect(self.dataFileCreated)
        
        self.StartScanButton.clicked.connect(self.startSliceScan)
        self.EndScanButton.clicked.connect(self.endScan)

    def dataFileCreated(self):
        self.filePath = os.path.join(dirname, "data/"  self.runNameBox.toPlainText() "_Rec.txt")
        print(self.filePath   " dataFileCreated")
        self.fs_watcher.addPath(self.filePath)
        self.fs_watcher.fileChanged.connect(self.update_graph)

    def update_graph(self):
        if exists(self.path):
            print("file exists!")
            #then read the filePath.txt and plots the data
    else:
        print("file doesn't exist yet")

    def endScan(self):
        #change some display things

    def runLongTask(self):
        # Step 2: Create a QThread object
        self.thread = QThread()
        # Step 3: Create a worker object
        self.worker = Worker()
        # Step 4: Move worker to the thread
        self.worker.moveToThread(self.thread)
        # Step 5: Connect signals and slots
        self.thread.started.connect(self.worker.run(param1,param2))
        self.worker.finished.connect(self.thread.quit)
        self.worker.finished.connect(self.worker.deleteLater)
        self.thread.finished.connect(self.thread.deleteLater)
        # Step 6: Start the thread
        self.thread.start()

if __name__ == "__main__":
    app = QtWidgets.QApplication(sys.argv)
    w = someWindow()
    w.show()
    sys.exit(app.exec_())

CodePudding user response：

As you explained, when your program starts, the file in question does not exist yet.

From the documentation of QFileSystemWatcher:

Adds path to the file system watcher if path exists. The path is not added if it does not exist, or if it is already being monitored by the file system watcher.

The solution to your problem is two-fold:

1. To get notified of a newly created file, add the parent directory (ie, the directory where you expect the file to appear) to your watcher (in your case, simply '.'). You will then get a notification (the directoryChanged signal) when any new files are created. Your update_graph method is already prepared for this dynamic as it checks for existence of your target file; so you can directly connect it to directoryChanged as you did with fileChanged.
2. As soon as the file is there, to get notified of any future changes to the file, add the file itself to the watcher like you already did, but this time also do it in update_graph (it may appear redundant, but note, as in the quote above, redundantly adding a file is not a problem; also when you add the path at start you cover the case where the file is already there).
3. In case the file is deleted, you will get the directoryChanged event again. You do not need to remove the path of the deleted file.
4. Finally, note that addPath() returns bool success. So you should check the return value in your program.

The reason for this behavior is that when you call addPath(), the file or directory found at that path is directly added to the watcher (not the path pointing to it). So the notification only works during a file's (or directory's) lifetime.