How to append a list of dataframes by removing header from all list and keeping only first header-CodePudding

I have a list of dataframes

li = [df1, df2,..]

All the dataframes in the list have common headers. I am appending the list of dataframes into a single df as follows:

path ="..."
all_files=glob.glob(path "*.csv")
all_files
li = []
for filename in all_files:
    df=pd.read_csv(filename,index_col=None,header=None)
    li.append(df)

However, will there be multiple headers after appending the list of dfs into one? If so, How to keep only the first header and remove the rest?

CodePudding user response：

Use pd.concat() For more info you can refer https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html

CodePudding user response：

You can simply concat in loop itself.

Let's assume csv1 looks like.

     Ref   tim
0    1    mow
1    2    pak
2    3    bow
3    4  tring

Let's assume csv2 looks like.

     Ref   tim
0    1    mow
1    2    pak
2    3    bow
3    4  tring

Read files uisng glob and concat at once as follows.

path =r"..."
all_files=glob.glob(path)
df = (pd.read_csv(each) for each in all_files)
df= pd.concat(df, ignore_index=True)
print(df)

Gives #

   Ref    tim
0    1    mow
1    2    pak
2    3    bow
3    4  tring
4    1    mow
5    2    pak
6    3    bow
7    4  tring