Python to go through multiple folders and process files inside them-CodePudding

I have multiple folders than contain about 5-10 files each. What I am trying to do is go to the next folder when finishing processing files from the previous folders and start working on the new files. I have this code:

for root, dirs, files in os.walk("Training Sets"): #Path that contains folders
    for i in dirs: #if I don't have this, an error is shown in line 4 that path needs to be str and not list
        for file in i: #indexing files inside the folders
            path = os.path.join(i, files) #join path of the files
            dataset = pd.read_csv(path, sep='\t', header = None) #reading the files
            trainSet = dataset.values.tolist() #some more code
            editedSet = dataset.values.tolist() #some more code
            #rest of the code...

The problem is that it doesn't do anything. Not even printing if I add prints for debugging.

CodePudding user response：

First off, be sure that you are in the correct top-level directory (i.e. the one containing "Training Sets". You can check this with os.path.abspath(os.curdir). Otherwise, the code does nothing since it does not find the directory to walk.

os.walk does the directory walking for you. The key is understanding root (the path to the current directory), dirs (a list of subdirectories) and files (a list of files in the current directory). You don't actually need dirs.

So your code is two loops:

>>> for root, dirs, files in os.walk("New Folder1"): #Path that contains folders
...     for file in files: #indexing files inside the folders
...             path = os.path.join(root, file) #join path of the files
...             print(path) # Your code here
...
New Folder1\New folder1a\New Text Document.txt
New Folder1\New folder1b\New Text Document2.txt