Try catch condition while combining CSV files in pandas-CodePudding

I am combining multiple csv files into a single dataframe using this line -

df = pd.concat(map(pd.read_csv, files), ignore_index=True)

I was earlier using a for loop where I combine two dataframes at time. This allowed me to use try-catch statements to catch any errors arising from empty/badly formatted csv files. But with the single-line command presented at the top, how do I enter a similar try-catch condition?

CodePudding user response：

try:

files = ['file1.csv', 'file2.csv', 'file3.csv']

def readcsv(path):
    try:
        dff = pd.read_csv(path)
    except pd.errors.EmptyDataError:
        print('error')
        dff = pd.DataFrame([]) #or anything else when error happen
        #I put empty dataframe here so the concat don't fail, but you can decide the behaviour you want when error happen (concat will fail if error happens...)
    return dff

df = pd.concat(map(readcsv, files), ignore_index=True)