Try/Except API Request in Tkinter still freezes sometimes?-CodePudding

The below is from a warehouse tool made to display KPI info. The program can run for a week fine, however freeze randomly, and I think it's because of an exception that is not being caught with the Try/Except.

I figured if I wrapped the entire lot in a Try/Except than it would catch all errors and just refresh after 20 seconds as expected. It does work for the majority of the time running.

Am I missing an important exception here?

def importData():
    now = datetime.now()
    date_time = now.strftime("%Y-%m-%d")
    yesterday = date.today() - timedelta(days=1)
    date_yesterday = yesterday.strftime("%Y-%m-%d")
    express_time = date.today() - timedelta(days=4)

    # --- Printed Data ---

    try:
      conn = http.client.HTTPSConnection()
      payload = ''
      headers = {
        'Content-Type': 'application/json',
        'StarShipIT-Api-Key': '',
        'Ocp-Apim-Subscription-Key': ''
      }
      conn.request("GET", "/api/orders?status=Printed&sort_column=ShippedDate&sort_direction=descending&include=packages&include=events&page_size=10", payload, headers )
      res = conn.getresponse()
    

      raw_data = res.read()
      decoded_data = raw_data.decode("utf-8")
      filtered_data = re.findall(r'(?<=order_id).*?(?=Generation)',decoded_data)
      order_number = re.findall(r'(?<=order_number":").*?(?=","reference)',decoded_data)

    

     # --- Shipped Data ---

      conn2 = http.client.HTTPSConnection()
      payload2 = ''
      headers2 = {
        'Content-Type': 'application/json',
        'StarShipIT-Api-Key': '',
        'Ocp-Apim-Subscription-Key': ''
      }
      conn2.request("GET", "/api/orders?status=Shipped&sort_column=ShippedDate&sort_direction=descending&include=packages&include=events&page_size=10", payload2, headers2 )
      res2 = conn2.getresponse()

      raw_data2 = res2.read()
      decoded_data2 = raw_data2.decode("utf-8")
      filtered_data2 = re.findall(r'(?<=order_id).*?(?=Generation)',decoded_data2)
      order_number2 = re.findall(r'(?<=order_number":").*?(?=","reference)',decoded_data2)

      stored_data.extend(x for x in filtered_data if x not in stored_data)
      stored_data.extend(x for x in filtered_data2 if x not in stored_data)
    
      order_number.extend(x for x in order_number2 if x not in order_number)

      pickle.dump(stored_data, open("stored_data.dat", "wb"))

      today_data_d = [ x for x in stored_data if "printed_date\":\"%s" % date_time in x ]
      today_data_c = [ x for x in today_data_d if "writeback_status\":\"Up to date" in x ]
      today_data_b = [ x for x in today_data_c if "writeback_details\":\"Success" in x ]
      today_data_a = [ x for x in today_data_b if "order_status\":\"Printed" in x ]
      today_data = [ x for x in today_data_a if "writeback_details\":\"Already fulfilled" not in x ]
    

      filter3_d = [ x for x in stored_data if "printed_date\":\"%s" % date_yesterday in x ]
      filter3_c = [ x for x in filter3_d if "writeback_status\":\"Up to date" in x ]
      filter3_b = [ x for x in filter3_c if "writeback_details\":\"Success" in x ]
      filter3_a = [ x for x in filter3_b if "order_status\":\"Printed" in x ]
      filter3 = [ x for x in filter3_a if "writeback_details\":\"Already fulfilled" not in x ]

      filter4 = [ x for x in filter3 if "printed_date\":\"%sT12" % date_yesterday in x ]
      filter5 = [ x for x in filter3 if "printed_date\":\"%sT13" % date_yesterday in x ]
      filter6 = [ x for x in filter3 if "printed_date\":\"%sT14" % date_yesterday in x ]
      filter7 = [ x for x in filter3 if "printed_date\":\"%sT15" % date_yesterday in x ]
      filter8 = [ x for x in filter3 if "printed_date\":\"%sT16" % date_yesterday in x ]
      filter9 = [ x for x in filter3 if "printed_date\":\"%sT17" % date_yesterday in x ]
      filter10 = [ x for x in filter3 if "printed_date\":\"%sT18" % date_yesterday in x ]
      filter11 = [ x for x in filter3 if "printed_date\":\"%sT19" % date_yesterday in x ]
      filter12 = [ x for x in filter3 if "printed_date\":\"%sT20" % date_yesterday in x ]
      filter13 = [ x for x in filter3 if "printed_date\":\"%sT21" % date_yesterday in x ]
      filter14 = [ x for x in filter3 if "printed_date\":\"%sT22" % date_yesterday in x ]
      filter15 = [ x for x in filter3 if "printed_date\":\"%sT23" % date_yesterday in x ]

      filter4.extend(filter5)
      filter4.extend(filter6)
      filter4.extend(filter7)
      filter4.extend(filter8)
      filter4.extend(filter9)
      filter4.extend(filter10)
      filter4.extend(filter11)
      filter4.extend(filter12)
      filter4.extend(filter13)
      filter4.extend(filter14)
      filter4.extend(filter15)

      today_data.extend(filter4)

      print(len(today_data))
    except: window.after(20000, importData)

CodePudding user response：

In general, it is not recommended to use a "naked" except. Instead, capture the info for the exception and log it, preferebly to a log file:

import logging
logging.basicConfig(
    filename='example.log',
    format='%(asctime)-15s %(levelname)s: %(message)s',
    encoding='utf-8',
    level=logging.DEBUG
)

def importData():
    try:
        # your code goes here
    except Exception as e:
        logging.error(e)
        window.after(20000, importData)

The reason for logging to a file is that with a tkinter program, you might not even see the messages on standard output.

If an exception occurs, you can at least see what happens in the log file.

And it would be a good idea to sprinkle some logging.debug calls throughout importData, so you can see what happens and inspect the time-stamps so you can see how long things take. Once the code works as expected, you can configure the logging level as INFO, and the debug calls will be ignored.

If you are calling importData code with after, its execution will interrupt the mainloop. And since network response times can be unpredictable and can take a while before they time-out, that could be the cause of the application freeze as well.

What I would suggest is to run importData as a second thread. Have it start a while-loop get the data (in a try/except) block and then sleep for a certain time before trying again.

If you are using Python 3 and if your tkinter is built with threading enabled, you should be able to update widgets from the second thread, since only one thread at a time can be executing Python bytecode. At the same time, network I/O releases the GIL so the second thread won't block the UI thread.

If you want to stop the second thread, simply set running to False from a callback in the main UI thread.

import threading
import time
import tkinter as tk

running  = True

def import_thread():
    while running:
        try:
            # fetch data
            # process data
            # update widgets
        except Exception as e:
            logging.error(e)
            # Maybe also write the error to a textbox?
        time.sleep(20)

if __name__ == "__main__":
    root = tk.Tk()
    # create window and widgets here
    it = threading.Thread(target=import_thread)
    # You could also start the thread from a menu or button callback.
    it.start()
    root.mainloop()