Copy 2nd row from all csv files (8000 files) located in a folder, and save it in new csv file includ-CodePudding

I have 8000 CSV files in a folder. All headers (i.e. 1st row) are same in all CSV files. I want to copy 2nd row of all csv files and paste in a new CSV file, to store in it including the header. New CSV file must be saved in the same folder. Please help using Python language.

I found a python program and it doesn't help much. Please help.

import csv

def load_data(filename)
     mylist = []
     with open(filename) as numbers:
          numbers_data = csv.reder(numbers, delimeter = '_')
          next()
          for row in numbers_data:
               mylist.append(row[1])
          return mylist

new_list = load_data('csv folder path location')

CodePudding user response：

Your code copy second column, not row.

You would rather need few next() without for-loop

header     = next(numbers_data) 
second_row = next(numbers_data) 

mylist.append(second_row)

But you would need for-loop to get filenames and run this function with different files

for filename in os.listdir(folder):

Full code (but not tested)

import csv

def get_second_row(filename)
    with open(filename) as fh:
        reader = csv.reader(fh)

        header     = next(reader)  # first row
        second_row = next(reader)  # second row

        return header, second_row

# --- main ---

folder = '/some/folder/with/csv'

header = None
all_rows = []

for filename in os.listdir(folder)
    if filename.lower().endswith('.csv'):
        full_path = os.path.join(folder, filename)
        header, row = get_second_row(full_path)
        all_rows.append(row)
    
# ---

with open('output.csv') as fh:
     writer = csv.writer(fh)

     # - write header -
     writer.writerow(header)     # `writerow` without `s` - to write single row

     # - write all rows at once -
     writer.writerows(all_rows)  # `writerows` with `s`   - to write list of rows