No such file or directory error in python while looping through several files-CodePudding

We are using a function to match a file pattern and if it matches we are running some jobs and eventually removing the file form the folder. But the file pattern match code is failing with "FileNotFoundError: [Errno 2] No such file or directory" when the for loop is running and some of the files are removed, this is happening in so less time that we are not able to replicate the issue in lower environment. Any idea how to catch the error or resolve it. In this case 'ABCD.XAF.XYJ250.A252.dat' file was present in the folder but removed as part of some other job.

def poke(self, context):
        full_path = self.filepath
        file_pattern = re.compile(self.filepattern)
        os.chdir(full_path)

        for files in sorted(os.listdir(full_path),key=os.path.getmtime):
            if not re.match(file_pattern, files):
                self.log.info(files)
                self.log.info(file_pattern)
            else:
                print("File Sensed " files)
                return True
        return False

Error:

 File "File_Sensor_Operator.py", line 25, in poke
    for files in sorted(os.listdir(full_path),key=os.path.getmtime):
  File "/usr/lib64/python3.6/genericpath.py", line 55, in getmtime
    return os.stat(filename).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: 'ABCD.XAF.XYJ250.A252.dat'

CodePudding user response：

IIUC, the error is happening because after the for loop starts, you have removed one of the files which means that it is no longer in the iterable returned bysorted(os.listdir(full_path),key=os.path.getmtime) which then throws the error.

So I think it should go away if you change that section of your code to this:

all_files = sorted(os.listdir(full_path),key=os.path.getmtime)
for files in all_files:
            if not re.match(file_pattern, files):

That way the iterable is assigned to a variable so the state is preserved as you loop through it.

CodePudding user response：

Your issue is that you collect all the files, delete one, then try to sort them by each files attributes (whereof one doesn't exist anymore)

import os
with open("dat.dat", "w "):
    pass
file_list = os.listdir(os.path.dirname(__file__))
os.remove("dat.dat")
for file_name in sorted(file_list, key=os.path.getmtime):
    print(file_name)

You can fix it by assigning the filenames and attributes to a variable before sorting.

import os
with open("dat.dat", "w "):
    pass
file_list = {
    filename:os.path.getmtime(filename) 
    for filename in os.listdir(os.path.dirname(__file__))
    if os.path.exists(filename)
}
os.remove("dat.dat")
for file_name in sorted(file_list.keys(), key=file_list.get):
    print(file_name)