Data not saved to disk after using socket timeout to break the loop in vscode-CodePudding

Objective

Get the data from a driving simulator via python socket and saved it to disk.

Code and Problem

I first click on the 'start drive' button on the simulator software that starts generating the data. Then, I run the following code in VSCode by clicking the 'Run Python File' that runs it in powershell in Windows.

import socket
import struct
import pandas as pd
from datetime import datetime

UDP_IP = "127.0.0.1"
UDP_PORT = 9000

sock = socket.socket(socket.AF_INET, # Internet
                     socket.SOCK_DGRAM) # UDP
sock.bind((UDP_IP, UDP_PORT))

appended_data = pd.DataFrame()

while True:
    
    data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
    fields = struct.unpack_from('=ddd', data)
    
    
    d = {'y': fields[0],
         'x': fields[1],
         'z': fields[2],
         'Time': datetime.now()}

    df1 = pd.DataFrame([d], columns=d.keys())
    
    appended_data = pd.concat([appended_data, df1])
    
    sock.settimeout(5)

# write DataFrame to an excel sheet 
appended_data.reset_index().to_excel('appended.xlsx')

Problem

The code runs successfully as you can see that the last two print outs are shown below. Then, I click on 'stop drive' on the simulator software that stops generating the data. That is why I timeout the socket so that it breaks the loop. But after the sock.settimeout(5), the python session ends and the last line appended_data.reset_index().to_excel('appended.xlsx') does not run. How can I make it run the last line so that the data is saved to an excel sheet?

             y             x         z                       Time
0  6374.120505  40150.996479 -1.735209 2022-04-21 08:06:16.888265
     y             x         z                       Time
0  0.0  40150.996479 -1.735209 2022-04-21 08:06:16.910364
Traceback (most recent call last):
  File "c:\Users\USERNAME\get_data_from_minisim.py", line 21, in <module>
    data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
socket.timeout: timed out
PS C:\Users\USERNAME>

EDIT: After going through the documentation and other stackoverflow questions, I came up with the following that was unsuccessful

appended_data = pd.DataFrame()

while True:
    try: 
        # time.sleep(5)
        data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
        fields = struct.unpack_from('=ddd', data)
        # print(fields[0],fields[1],fields[2])
        
        d = {'y': fields[0],
            'x': fields[1],
            'z': fields[2],
            'Time': datetime.now()}

        df1 = pd.DataFrame([d], columns=d.keys())
        print(df1)
        appended_data = pd.concat([appended_data, df1])
        # print(appended_data)
    
    except:
        sock.settimeout(5)
        appended_data.reset_index().to_excel('appended.xlsx')

I also tried:

except sock.settimeout(5) as e:
        if e == "timed out":
            appended_data.reset_index().to_excel('appended.xlsx')

and:

except sock.timeout:
        # write DataFrame to an excel sheet 
        appended_data.reset_index().to_excel('appended.xlsx')

** But nothing worked.** Specifically, the loop is not broken after clicking 'stop driving' in these later attempts.

Could you please guide me how can I break the loop and save the appended_data to excel?

CodePudding user response：

Assuming the socket code is correct this should save your DataFrame when your connection times out:

import socket
import struct
import pandas as pd
from datetime import datetime

UDP_IP = "127.0.0.1"
UDP_PORT = 9000

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)  # Internet  # UDP
sock.bind((UDP_IP, UDP_PORT))

appended_data = pd.DataFrame()

while True:
    try:
        data, addr = sock.recvfrom(1024)  # buffer size is 1024 bytes
        fields = struct.unpack_from("=ddd", data)

        d = {"y": fields[0], "x": fields[1], "z": fields[2], "Time": datetime.now()}
        # Also you can clean up your code as you can append dict to your DataFrame, so you can omit creation of "df1"
        appended_data = appended_data.append(d, ignore_index=True)

        sock.settimeout(5)
    except socket.timeout as timeout_exception:
        print(f"Lost communication with remote\n{timeout_exception}")
        break

# write DataFrame to an excel sheet
appended_data.reset_index().to_excel("appended.xlsx")

Additional note to make your code future proof, examples in documentation shows that you should use socket inside with statement, which is a good practice as you don't have to manually close the connection when you are finished.