Home > Software engineering >  Pyspark : How to concat two dataframes in Pyspark
Pyspark : How to concat two dataframes in Pyspark

Time:04-13

df_1 :

NBB1
776

And df_2

NBB2
4867

I will to obtain this dataframe in Pyspark df :

NBB1 NBB2
776 4867

CodePudding user response:

You need to perform a crossJoin between the two dataframes. See below for details -

from pyspark.sql import Row

df1 = spark.createDataFrame([Row(NBB1 = 776)])
df1.show()
#Output
 ---- 
|NBB1|
 ---- 
| 776|
 ---- 

df2 = spark.createDataFrame([Row(NBB2 = 4867)])
df2.show()
#Output
 ---- 
|NBB2|
 ---- 
|4867|
 ---- 


df1.crossJoin(df2).show()
#Output
 ---- ---- 
|NBB1|NBB2|
 ---- ---- 
| 776|4867|
 ---- ---- 

  • Related