Lets say I have a dataframe as following:

| id |col 1       |col 2

| 1  | ["A","B"]  |["C","D"]

| 2  | ["A","B"]  |["X","Y"]

| 3  | ["B","D"]  |["N","M"]

I need an output as:

| id |merged(col1 and col2)

| 1  | ["A","B","C","D"]

| 2  | ["A","B","X","Y"]

| 3  | ["B","D","N","M"]

CodePudding user response：

You can use the concat function.

df = df.select('id', F.concat('col1', 'col2').alias('merged(col1 and col2)'))
df.show(truncate=False)

CodePudding user response：

Yes, you can use apply for that -

df['merged'] = df.apply(lambda x: x["col1"]   x["col2"], axis=1)