Remove filler spaces from blank lines in linux script-CodePudding

I am trying to work on a bash script that will take files from one github repo and copy them over to another one.

I have this mostly working however 1 file I am trying to move over has spaces on all of its blank lines like so:

FROM metrics_flags ORDER BY DeliveryDate ASC
)
                        
SELECT * FROM selected;
""";

Notice how its not just a blank line, there are actually 10-20 spaces in between the 2 blocks of code on that blank line. Is there some unix command that can parse the file and remove the spaces (but keep the blank line)?

I tried

awk 'NF { $1=$1; print }' file.txt

and

sed -e 's/^[ \t]*//' file.txt

with no success.

CodePudding user response：

Odd ...

sed -i 's/^[[:space:]]*$//g' file.txt

definitely works for me; I don't see why your sed version wouldn't, though.

CodePudding user response：

awk used without changing delimeters splits records (lines) into white-space-separated fields. By default any print commands obey the same separators for the output but any empty fields can be removed resulting in their white-space-separators not being used.

The 'trick' is to get awk to re-evaluate the line by setting any field (even empty ones) to itself:

awk '{$1=$1; print}' test.txt

will remove all white space that is not surrounding other printable characters and return the file contents to stdout where it can be redirected to file if required.

I don't know why you used NF as a pattern in your awk attempt, nor why it caused it to fail, but the similar approach without it, as above, works fine.

edit after a quick experiment, I think what was happening with your awk attempt was that setting the pattern to NF caused awk to skip lines with no printable fields completely. Removing that pattern allows the now empty lines to be printed.

CodePudding user response：

This should do what you describe, replacing leading whitespace only from empty lines:

sed -E 's|^\s $||' file

The -E (extended regex) is required for \s (\t also), meaning one or more whitespace characters. I think you might have accidentally used a lower e.

If you like the output, you can add -i to apply the edit to your file.

This is an example of using awk to achieve the same:

awk '{gsub(/^\s $/, "")}; { print }' file

To apply it, use -i inplace:

awk -i inplace '{gsub(/^\s $/, "")}; { print }' file

I tested this on Ubuntu 22.04 with GNU sed 4.8 and GNU awk 5.1.0