I need to achieve the following in my python module.

1. Get a directory as an argument through the argparse module
2. Use that directory to iterate through all the files present in that directory.
3. Later, append only the CSV files to a dictionary and store them there.

I have tried os.walk, but it does not return anything.

I have written code to get a directory as input and then iterate through it for files. But it throws errors.

Below is the code

 import argparse
 import os
 import re
    
 # Accepting arguments for source_directory, my_sql connection details and table name to 
   update the MySQL entries to
    
parser = argparse.ArgumentParser(
        description='Utility to update MySQL tables on remote host')
parser.add_argument('-s', '--source_dir', help='Enter Path For CSV File')
    
args = parser.parse_args()
    
# Empty dictionary to store list CSV files in directory
files_in_dir = {}
    
# Function to iterate through the source_directory and read-only CSV files and append them 
   to files_in_dir dictionary
    
if args.source_dir:
    for file in args.source_dir:
    
        if os.path.abspath(file.name).endswith('.csv'):
            files_in_dir[(os.path.abspath(file.name))] = 0
    
        else:
            print("The file %s is not a .csv file and it will be ignored" % (file.name))
    
print(files_in_dir)

And this is the error when executed through the terminal.

 python3 sourcecheck.py -s /home/thepredator/Desktop/VOIS/csv_files/*
usage: sourcecheck.py [-h] [-s SOURCE_DIR]
sourcecheck.py: error: unrecognized arguments: /home/thepredator/Desktop/VOIS/csv_files/Book1.csv /home/thepredator/Desktop/VOIS/csv_files/Build.py /home/thepredator/Desktop/VOIS/csv_files/car_data.csv /home/thepredator/Desktop/VOIS/csv_files/db.py /home/thepredator/Desktop/VOIS/csv_files/Edited.csv /home/thepredator/Desktop/VOIS/csv_files/Iris.csv /home/thepredator/Desktop/VOIS/csv_files/Lab_Python_for_Data_Analytics.ipynb /home/thepredator/Desktop/VOIS/csv_files/startupNaN.csv /home/thepredator/Desktop/VOIS/csv_files/startupog.csv /home/thepredator/Desktop/VOIS/csv_files/text.csv

I have written the above code by watching different videos. Please help me fix the errors. Also, I'm not a python expert.

CodePudding user response：

I changed files_dir = {} line to

CodePudding user response：

You can use this code`

import os

files = os.listdir("/path_to_directory")    

csv = list(filter(lambda f: f.endswith('.csv'), files))

csv has all the files with .csv extension