How to read pairwise csv and json files having same names inside a folder using python?-CodePudding

Consider my folder structure having files in these fashion:-

abc.csv
abc.json
bcd.csv
bcd.json
efg.csv
efg.json

and so on i.e. a pair of csv files and json files having the same names, i have to perform the same operation by reading the same named files , do some operation and proceed to the next pair of files. How do i go about this? Basically what i have in mind as a pseudo code is:-

for files in folder_name:
    df_csv=pd.read_csv('abc.csv')
    df_json=pd.read_json('abc.json')
    # some script to execute
    #now read the next pair and repeat for all files

CodePudding user response：

Did you think of something like this?

import os

# collects filenames in the folder
filelist = os.listdir()
# iterates through filenames in the folder
for file in filelist:
# pairs the .csv files with the .json files
    if file.endswith(".csv"):
        with open(file) as csv_file:
            pre, ext = os.path.splitext(file)
            secondfile = pre   ".json"
            with open(secondfile) as json_file:
                # do something

CodePudding user response：

If the directory structure is consistent you could do the following:

import os

for file_name in {x.split('.')[0] for x in os.listdir('./path/to/dir')}:
    df_csv = pd.read_csv("{f_name}.csv")
    df_json = pd.read_json("{f_name}.json")
    # execute the rest

CodePudding user response：

You can use the glob module to extract the file names matching a pattern:

import glob
import os.path

for csvfile in glob.iglob('*.csv'):
    jsonfile = csvfile[:-3]   'json'
    # optionaly control file existence
    if not os.path.exists(jsonfile):
        # show error message
        ...
        continue
    # do smth. with csvfile
    # do smth. else with jsonfile
    # and proceed to next pair