Most efficient way to append rows from one dataframe to another-CodePudding

import numpy as np
import pandas as pd

data2 = [['Name 1', 20, 20, 15, 20], ['Name 2', 15, 10, 10, 8]]
df2 = pd.DataFrame(data2, columns=['Name', 1, 2, 3, 4])

data1 = [['New 1', 5, 10, 15, 20], ['New 2', 2, 5, 10 , 20]]
df = pd.DataFrame(data1, columns=['Name', 1, 2, 3, 4])

print(df)

    Name  1   2   3   4
0  New 1  5  10  15  20
1  New 2  2   5  10  20

new_line = df.iloc[0]
df2 = df2.append(new_line)

print(df2)

     Name   1   2   3   4
0  Name 1  20  20  15  20
1  Name 2  15  10  10   8
0   New 1   5  10  15  20

I have the desired result I want. However, when I loop through this hundreds of times increasing the df.iloc[1], [2], etc it seems to bottleneck my for loop. Is there a more efficient way to select a row of df then append it to df2?

CodePudding user response：

df2.merge(df,how="outer")

This would merge df to df2.

CodePudding user response：

If you want to append the whole dataframe something like this should do the trick:

df.append(df2)

But I am assuming you know this already, and only want a selected few specified rows. In that case, I would create an array with the desired rowsID's from df2 that you want to append to df.

So something like this should work:

rowidarr = []
for items in df.iterrows():
    ###condition to get rowid you want
    rowidarr.append(index)
filter_df  = df[df.index.isin(rowidarr)]
df.append(filter_df)