How to extract data from Arduino only when it is available and save the data?-CodePudding

I am trying to extract sensor data from Arduino. Arduino code sends sensor data using serial communication every 1 second. My goal is to display this data and plot it in real-time in a GUI using Tkinter. But where I seem to be stuck is retrieving data from Arduino. In my code, I tried to extract the data and save them in multiple variables. But when Arduino is sending only 5 seconds worth of data which is 5 sensor measurements, my python code is saving more than 60 measurements in the variable. I don't have any idea how to solve this. I have been stuck here for some time. Here is my python code.

In the given pictures I ran the code for 5 seconds. That is 5 measurements will be sent from Arduino. I am recieving the data correctly, but when I process them and seperate them in individual variables I have analog_voltage containing:80 , current:70, voltage:75 and temperature:65 measurements as they should only contain 5. What should I do, is there any way to go around this problem?

This is the same issue when I am trying to save the file. For the 5 seconds of data the program saves a million rows of data in csv. Any help is much appreciated. Thank you for reading.

The image for arduino data is just for reference as it is not possible to run both the programs in a same time due to the serial communication protocols hence the data looks different.

data_to_sort =[]
analog_voltage = []
Voltage= []
Current=[]
Temperature=[]

def arduino_read_serialdata():
    global ser
    ser = serial.Serial('COM4', 9600, timeout=1)
    time.sleep(1.0)
    ser.flushInput() 
    while True:
        global analog_voltage, Voltage, Temperature, Current, yp, cleaned_data, anvol
        incoming_data = ser.inWaiting()
        incoming_data = ser.readline()
        incoming_data = incoming_data.decode()
        cleaned_data = incoming_data.strip("/n")
        data_to_sort.append(cleaned_data)
  
        
        for everyrow in data_to_sort:
        
            if everyrow.startswith("Measuring voltage and current with INA219 ..."):
                pass

            elif everyrow.startswith("MAX31855 test"):
                pass
           
            elif everyrow.startswith("Initializing sensor...DONE checking reading the values now."):
                pass
           
            elif everyrow.startswith("The Analog Voltage is:"): 
                anvol = everyrow[-8:]
                anvol = float(anvol)
                analog_voltage.append(anvol)
            
            elif everyrow.startswith("Bus Voltage............:"):
                vol = everyrow[-8:]
                Voltage.append(float(vol))
           
            elif everyrow.startswith("Current..............:"):
                cur = everyrow[-8:]
                Current.append(float(cur))
           
            elif everyrow.startswith("Temperature in celcius:"):
                temp = everyrow[-7:]
                Temperature.append(float(temp))
            else:
                pass

    
        with open("Sensor_data.csv", 'w ', newline='') as output_file:
            output_file.flush()
            output_file = csv.writer(output_file)    
            fields = ['Analog Voltage V', 'Voltage V', 'Current mA', 'Temperature °C']        
            output_file.writerow(fields)
            for idx in range(0, len(analog_voltage)):
                output_file.writerow([analog_voltage[idx]])    
       if keyboard.is_pressed("q"):
            print("Manually interrupted by user")
            ser.close()
            # output_file.close()
            print("Serial port closed")
            file.close()
            print("File closed, Writing done")

CodePudding user response：

You have an array

data_to_sort =[]

and you append each incoming line to it

data_to_sort.append(cleaned_data)

And with each incoming data, you process the whole data

for everyrow in data_to_sort:

and add the values to these four lists:

analog_voltage = []
Voltage= []
Current=[]
Temperature=[]

and these lists are never cleared.

What should be done instead:

either clear these four lists with each incoming data or
process only the incoming line

My personal preference would be the latter. Instead of always writing the whole output file, append only the last line to the output file using "a" instead of "w ".