Fetch unique values from first columns and concat the values of other columns using spark dataframe-CodePudding

I have a spark dataframe with two columns I need to fetch unique values from first columns and concat the values of other columns

for ex: enter image description here

I wanted output as below: enter image description here

how to achieve this using spark dataframe?

CodePudding user response：

You can provide these simple script in your question. It will be answered more easy.

DECLARE @Table TABLE (
    column1 sysname,
    column2 CHAR(1)
);

INSERT @Table (column1, column2)
VALUES ('D112', 'a'),
    ('D232', 'b'),
    ('D112', 'c'),
    ('D334', 'd'),
    ('D232', 'e'),
    ('D112', 'f');

AND THEN here is the answer:

SELECT t.column1, STRING_AGG(t.column2, ',')
FROM @Table AS t
GROUP BY t.column1;

CodePudding user response：

Try this:

df.groupBy($"column1").agg(concat_ws(",", collect_list($"column2")))