I want to open a variable number of csv files and then I would like to iterate over the csv files opened and upload 1 row of each file at a time to my sql database. For example, loop through each file uploading the first row of each file to the database, then loop again through each file uploading the second row of each file to the database. However, I'm stuck in having the csv files ready to be uploaded in a single object. The error happens at 'csv_data[i] = csv.reader...' Each file is for a different table, so I cannot append them.

import csv
import sys

i = 0
for argv in sys.argv[1:]:
    csv_file = open(argv, newline='', encoding='utf-8-sig')
    csv_data[i] = csv.reader(csv_file, dialect='excel', delimiter=',', quotechar='|')
    csv_file.close()
    i  = 1

After this code, I would need something to loop through each file uploading a certain row number.

CodePudding user response：

zip together the files, iterate through them:

file_handles = [open(file, newline='', encoding='utf-8-sig') for file in argv[1:]]
readers =  (csv.reader(file, dialect='excel', delimiter=',', quotechar='|') for file in file_handles)

# zip here
for line_group in zip(*readers):
    # line_group is a tuple of line i of each file


# don't forget to close your files
for file_handle in file_handles:
    try:
        file_handle.close()
    except:
        print("Issue closing one of the files")