Linux shell script to read from a file with separator and then manipulate some fields and redirect w-CodePudding

Need to read line by line a file example below and then get only and put it in separate file file 1

1~abc~select col1,col2,col3,col4 from <tablename> where <condition> group by~xyz
2~abc~select col1,col2,col3      from <tablename> where <condition> group by~xyq
3~abc~select col1,col2           from <tablename> where <condition> group by~xyg

new file expecting

1~abc~<condition>~xyz
2~abc~<condition>~xyq
3~abc~<condition>~xyg

is there any simpler way to get this done using shell script.

CodePudding user response：

This can be achieved using sed utility in unix. A single expression is sufficient actually sed -e 's/select .* where //;s/ group by//'

See it in action here:

pankaj@pankaj-mac TP % cat inputfile.txt 
1~abc~select col1,col2,col3,col4 from <tablename> where <condition> group by~xyz
2~abc~select col1,col2,col3      from <tablename> where <condition> group by~xyq
3~abc~select col1,col2           from <tablename> where <condition> group by~xyg

pankaj@pankaj-mac TP % sed -e 's/select .* where //;s/ group by//' inputfile.txt > newfile.txt
pankaj@pankaj-mac TP % cat newfile.txt 
1~abc~<condition>~xyz
2~abc~<condition>~xyq
3~abc~<condition>~xyg

pankaj@pankaj-mac TP %

CodePudding user response：

In the slightly more general case where the regexes could also match within some of the other fields, Awk might be a better choice than sed.

awk -F '~' '{
  sub(/.* where /, "", $3);
  sub(/ group by.*/, "", $3)
 }1' file >newfile

The final 1 is the common Awk idiom for "print the current input unconditionally".

This could also be accomplished with a while read -r loop in the shell, but you generally want to avoid that; see Bash while read loop extremely slow compared to cat, why?