remove duplicateds words for multiple strings in bash-CodePudding

I would like to know how can I remove duplicate words from every line in bash with sed, awk, etc.. I have this file with 2000 lines, and I would like to know how can I keep one uniq word per line

OG0000005 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373 K00373  K00373
OG0000006 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374 K00374  K00374
OG0000007 K03089  K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089 K03089
OG0000008 K15554  K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599 K15599
OG0000009 K15555  K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555 K15555
OG0000010 K00817  K09758 K00817 K00817 K00817 K00817 K00817 K00817 K00817 K00817 K00817 K00817 K00817 K00817 K00817 K00817 K00817
OG0000011 K07267  K07267  K07267  K07267 K07267 K07267 K07267 K07267 K07267 K07267 K07267 K07267 K07267 K07267 K07267 K07267
OG0000012 K22397  K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714 K01714
OG0000013 K00370  K07812 K00370 K00370 K00370 K00370 K00370 K00370 K00370 K00370 K00370 K00370 K00370 K00370 K00370 K00370 K00370

so the output should be like

OG0000005 K00373
OG0000006 K00374
OG0000007 K03089  
OG0000008 K15554  K15599 
OG0000009 K15555 
OG0000010 K00817  K09758

I tried with
sort file | uniq

wile read line
do
sort && uniq
done < file

CodePudding user response：

You may use this awk solution:

awk '
{
   delete seen
   printf "%s", $1
   for (i=2; i<=NF;   i)
      if (!seen[$i]  )
         printf "%s", OFS $i
   print ""
}' file

OG0000005 K00373
OG0000006 K00374
OG0000007 K03089
OG0000008 K15554 K15599
OG0000009 K15555
OG0000010 K00817 K09758
OG0000011 K07267
OG0000012 K22397 K01714
OG0000013 K00370 K07812