Remove Duplicates numbers in Shell Script-CodePudding

I have a file called Sparrow.txt and its content is follows

ENV-NAME|ACTIVE-SITE|MCM|BDD|ELOW|OPT|MSP
======|===========|========|========|=======|=========|========
EAST| PROD | 2311| 2312 2312 | 2313 2313 | 2314 2314 | 2315 2315
EAST| STAGE | 2311 | 2312 2312 | 2313 2313 | 2314 2314 | 2315 2315

I want to remove the duplicates in the file and following should be the output

ENV-NAME|ACTIVE-SITE|MCM|BDD|ELOW|OPT|MSP
======|===========|========|========|=======|=========|========
EAST| PROD | 2311| 2312 | 2313 | 2314 | 2315 
EAST| STAGE | 2311 | 2312 | 2313 | 2314 | 2315

I have tried to sort and uniq but that didn't work

CodePudding user response：

If you can use perl :

perl -pe 's|\b([0-9] ) \1\b|\1|g' Sparrow.txt

CodePudding user response：

$ cat tst.awk
BEGIN { FS=OFS="|" }
{
    for (i=1; i<=NF; i  ) {
        if ( split($i,t," ") > 1 ) {
            $i = " " t[1] " "
        }
    }
    print
}

$ awk -f tst.awk file
ENV-NAME|ACTIVE-SITE|MCM|BDD|ELOW|OPT|MSP
======|===========|========|========|=======|=========|========
EAST| PROD | 2311| 2312 | 2313 | 2314 | 2315
EAST| STAGE | 2311 | 2312 | 2313 | 2314 | 2315

CodePudding user response：

If you've got GNU Sed, try:

sed 's/\<\([[:digit:]]\ \)[[:space:]]\ \1\>/\1/g' Sparrow.txt