Home > other >  How to write data to a file continuously without blocking main loop in Python
How to write data to a file continuously without blocking main loop in Python

Time:10-22

I am new to Python and I have a scenario where I want to write measurement data to 2 files, one file within the main() function and another file within a function WriteToFile() which writes a new line of data every 10 seconds. How can I make WriteFile() get fired once in the main() loop and then run until 2 minutes after the main() loop iteration completes? How can I then exit WriteToFile() gracefully in CleanUp(), ensuring that FastFile.txt has closed properly? Here is a simplified program example:

from datetime import datetime
import os
import time
import sys
import random

M_Max = 2000
M_Array = [740] 
D_Array = [5, 10, 20, 100, 150, 200, 400,600, 740]  



M=random.random()*20000
D=random.random()*7500


#write data to a file every ten seconds  
def WriteToFile():  
    while True:
        h = open("FastFile.txt", "a")
        current_time_t=datetime.now().strftime("%d.%m.%y %H:%M:%S")
        p = random.random()*1000
        c = random.random()*100
        
        print("Fast_Time :", '{} {:9.0f} {:9.0f} {:9.3f} {:9.1f}  '.format(current_time_t, M, D, p, c))

        h.write("\n %s , %s , %s , %s, %s  " % (current_time_t, str('{:.1f}'.format(M)), str('{:.1f}'.format(D)), str('{:.3f}'.format(p)), str('{:.2f}'.format(
        c))))
        h.close()
        time.sleep(10)

    
        
    

def CleanUp():
    print("Cleaning up")
    
    #wait two minutes then stop WriteToFile()
    time.sleep(120)
    sys.exit()
    
    print("Done")
    


def main():
    global  M, D
   
    WriteToFile()  # call this function and start writing data including the global variables M and N generated here

    print("Starting measurement")
    time.sleep(3)
    
    for M in M_Array:

        for D in D_Array:  
            
            p = random.random()*1000
            c = random.random()*100

            g = open("SlowFile.txt", "a")
            

            current_time_t = datetime.now().strftime("%d.%m.%y %H:%M:%S")

            print("Main_Time :", '{} {:9.0f} {:9.0f} {:9.3f} {:9.1f}  '.format(current_time_t, M, D, p, c))
            
            #write the result of each iteration to a file
            g.write("\n %s , %s , %s , %s, %s  " % (current_time_t, str('{:.1f}'.format(M)), str('{:.1f}'.format(D)), str('{:.3f}'.format(p)), str('{:.2f}'.format(
                c))))
            g.close()


    CleanUp()


if __name__ == "__main__":
    main()

I am using Python 3.5.3 but can upgrade to accommodate solutions that require higher versions of Python.

CodePudding user response:

You need threading! Your if __name__="__main__" has to be like below:

if __name__ == "__main__":
    t1 = threading.Thread(target=WriteToFile)
    t2 = threading.Thread(target=loop) # put the loop in a new function and call it using this
    t1.start()
    t2.start()
    t1.join()
    t2.join()

Your main() function has to be renamed to loop or anything you like, without the call to WriteToFile(). Remember to import threading at the beginning of the code.

  • Related