How Can i "stick" two or more pd dfs together-CodePudding

Lets say I have three dfs

x,y,z
0,1,1,1
1,2,2,2
2,3,3,3

a,b,c
0,4,4,4
1,5,5,5
2,6,6,6

d,e,f
0,7,7,7
1,8,8,8
2,9,9,9

How can I stick them all together so that i get:

x,y,z
0,1,1,1
1,2,2,2
2,3,3,3
a,b,c
0,4,4,4
1,5,5,5
2,6,6,6
d,e,f
0,7,7,7
1,8,8,8
2,9,9,9

I am not fussed if it's in a df or not hence I haven't included a new index. Essentially I just want to glue n amount of dfs together to save me having to copy and paste the data into an excel sheet myself.

CodePudding user response：

If you want to save all your dataframes in the same file one after the other, use a simple loop with to_csv and use the file append mode (a):

dfs = [df1, df2, df3]

for d in dfs:
    d.to_csv('out.csv', mode='a')

NB. the initial file must be empty or non existent

output out.csv:

,x,y,z
0,1,1,1
1,2,2,2
2,3,3,3
,a,b,c
0,4,4,4
1,5,5,5
2,6,6,6
,d,e,f
0,7,7,7
1,8,8,8
2,9,9,9

CodePudding user response：

You might try this:

start_df = df1
other_dfs = [df2, df3]
new_df = pd.concat([start_df, *[x[::-1].append(dict(zip(x.columns, x.columns)), ignore_index=True)[::-1].reset_index(drop=True).set_axis(start_df.columns, axis=1) for x in other_dfs]], ignore_index=True)

Output:

>>> new_df
    x  y  z
0   1  1  1
1   2  2  2
2   3  3  3
3   a  b  c
4   4  4  4
5   5  5  5
6   6  6  6
7   d  e  f
8   7  7  7
9   8  8  8
10  9  9  9

Basically what we're doing here is, for each df in other_dfs, we reverse it, append the columns of that dataframe to the end, re-reverse it, change its columns to match start_df's columns, and then we concatenate them all together.

CodePudding user response：

Have a look at the to_csv() method of DataFrame.

print(df1.to_csv(index=False))
print(df2.to_csv(index=False))
print(df3.to_csv(index=False))

That being said, there is a to_excel() method as well that may solve the potential XY Problem referenced in the comments.

CodePudding user response：

Here is an example using 'concat'

import pandas as pd
import matplotlib.pyplot as plt

d = {'number_order' : [1, 2, 3,
                       4, 5, 6],
     'client' : ['x', 'y', 'z',
'd', 'e', 'f'],
'value' : ['120', '187.74', '188.7', '300', '563.2', '198.0']
     }
d1 = {'number_order' : [1, 2, 3,
                       4, 5, 6],
     'client' : ['a', 'b', 'c',
'd', 'e', 'f'],
'value' : ['120', '187.74', '188.7', '300', '563.2', '198.0']
     }
d2 = {'number_order' : [1, 2, 3,
                       4, 5, 6],
     'client' : ['a', 'b', 'c',
'd', 'e', 'f'],
'value' : ['120', '187.74', '188.7', '300', '563.2', '198.0']
     }


     
df = pd.DataFrame(data = d)
df1 = pd.DataFrame(data = d1)
df2 = pd.DataFrame(data = d2)
f = [df,df1,df2]
df = pd.concat(f)
print(df)