How to unique by column large file via sort command?-CodePudding

Home > Enterprise > How to unique by column large file via sort command?

How to unique by column large file via sort command?

Time：09-29

i have large list (6 million lines) and want to sort unique by column #2, how to do it by sort command? In example

CONT,000-00-0000,GRAM
BEVE,507-66-6876,IGHT
MICH,000-00-0000,EVINS
CONT,111-11-1111,GRAM

trasnfer to

CONT,000-00-0000,GRAM
BEVE,507-66-6876,IGHT
CONT,111-11-1111,GRAM

CodePudding user response：

$ awk -F, '!array[$2]  ' input_file
CONT,000-00-0000,GRAM
BEVE,507-66-6876,IGHT
CONT,111-11-1111,GRAM