I have two lists whose elements are dataframes and both the lists are of equal lengths.

df_list1=[]

df_list2=[]

I want to merge the dataframes from two lists and put it in a new list

new_list=[]

for i in df_list1:
    new_list.append(pd.concat([df_list1[i],df_list2[i]]))

I was wondering if there is an efficient(faster) way to do this.

CodePudding user response：

List comprehension is a little faster, but the gain in execution time is very minimal.

I compared the following with 2 identical lists of 6 identical DataFrames.

For loop:

for i in range(len(list1)):
    new_lst.append(pd.concat([list1[i], list2[I]]))

List Comprehension:

new_lst = [pd.concat([df1, df2]) for df1, df2 in zip(list1, list2)]

I ran them 5 times in a row. The table below tallies the runtime in seconds.

I have no idea why the large difference in the first execution but a few more iterations resulted in similar patterns we see in the last 3 rows.