I have a set of Excel files in a folder that I am running through different functions in a large Python script. We have set up exceptions to handle issues with the formatting in any Excel file, however if one of the files throws an exception, none of the other ones get "processed." Is there a way to continue running code in Python for other files that don't throw any exceptions? Here is the main function we are using to run the files through all the other functions. Most of the functions listed here contain additional function calls to validate the format of the files (number of columns, names of columns, checking for mistaken duplicate rows, etc.) where we have built in exceptions for formatting discrepancies that stop the code if present.

def validation_and_processing(file_list):
    for file in files:
        if file.startswith('Nomura'):
            check_filename_format(input_file = file)
            nomura = nomura_validation(input_dir = input_dir, file = file)
            nomura_df = nomura_processing(nomura)
            write_output(path=output_dir, filename=file, dataframe=nomura_df)
            shutil.move(input_dir file, archive_dir file)   
            
        elif file.startswith('MSTN'):
            check_filename_format(input_file = file)
            morgan = morgan_validation(input_dir = input_dir, file = file) 
            morgan_df = morgan_processing(morgan)
            write_output(path=output_dir, filename=file, dataframe=morgan_df)
            shutil.move(input_dir file, archive_dir file)    
            
        elif file.startswith('JPM'):
            check_filename_format(input_file = file)
            jpm, input_file = jpm_validation(input_dir = input_dir, file = file)
            jpm = jpm_processing(jpm, input_file)
            write_output(path=output_dir, filename=file, dataframe=jpm)
            shutil.move(input_dir file, archive_dir file)
                
        elif file.startswith('BofA'):
            check_filename_format(input_file = file)
            bofa_not_ginnie, bofa_ginnie = bofa_validation(input_dir=input_dir, file=file)
            bofa_final = bofa_processing(df1=bofa_not_ginnie, df2=bofa_ginnie, input_file=file)       
            write_output(path=output_dir, filename=file, dataframe=bofa_final)
            shutil.move(input_dir file, archive_dir file)
        
        elif file.startswith('CITI'):
            check_filename_format(input_file = file)
            citi = citi_validation(input_dir=input_dir, file=file)
            citi_final = citi_processing(df=citi, input_file=file)    
            write_output(path=output_dir, filename=file, dataframe=citi_final)
            shutil.move(input_dir file, archive_dir file)

files = os.listdir(input_dir)
validation_and_processing(file_list = files)

So for example, if the BofA file has mis-formatted columns, then an exception will be raised, the code will stop running, and none of the other files in the directory will be processed, even if they have no problems with their formatting. I also noticed that the order of the files in the directory doesn't matter. Even if the BofA file is the last one, none of the other ones will complete their processing, which I also don't understand because based on the order in this for loop and if statements, I would assume all the files except for CITI should be processed right?

CodePudding user response：

You need to add a try/except block to your code to handle errors.

def validation_and_processing(file_list):
    for file in files:
        try:
            if file.startswith('Nomura'):
                ...   
            
            elif file.startswith('MSTN'):
                ...    
            
            elif file.startswith('JPM'):
                ...
                
            elif file.startswith('BofA'):
                ...
        
            elif file.startswith('CITI'):
                ...
        except:
            print("Error with file:", file)

files = os.listdir(input_dir)
validation_and_processing(file_list = files)