replace pattern with newline in shell variable-CodePudding

A script save.sh uses 'cp' and outputs its cp errors to an errors file. Mostly these errors are due to the origin filesystem being EXT4 and the destination filesystem being NTFS or FAT and doesnt accept some specia characters.

Another script one rrors.sh reads the error file so as to best manage files that could not be copied : it copies them toward a crafted filename file that's OK for FAT and NTFS = where "bad" characters have been replaced with '_'. That works fine for allmost all of the errors, but not 100%.

In this error file, the special characters in the filenames seem to be multiple times escaped :

simple quotes ' appear as '\''.
\n (real newline in filenames!) appear as '$'\n'' (7 glyphs !)

I want to unescape these so as to get the filename.

I convert quotes back to ' with line=${line//\'\\\'\'/\'}. That's OK.

But how can i convert the escaped newline back to a real unescaped \n in $line variable = how can i replace the '$'\n'' to unescaped \n in variable ?

The issue is not in recognising the pattern but in inserting a real newline. I've not been able to do it using same variable expansion syntax. What other tool is advised or other way of doing it ?

CodePudding user response：

The question is:

how can i replace the '$'\n'' to unescaped \n in variable ?

That's simple:

var="def'$'\n''abc"
echo "${var//\'$\'\\n\'\'/$'\n'}"

I think I remember, that using ANSI C quoting inside variable expansion happened to be buggy in some version of bash. Use a temporary variable in such cases.

What other tool is advised or other way of doing it ?

For string replacement in shell, the most popular tools are sed (which the name literally comes from "String EDitor") and awk. Writing a parser is better done in full-blown programming languages, like Python, C, C and similar.

The only way to decode cp output correctly, is to see cp source code, see how it quotes the filenames, and decode it in the same way. Note that it may change between cp flavors and versions, so such a tool may need to query cp version and will be not portable.

Note that parsing cp output is a very very very very bad idea. cp output is in no way standardized, may change anytime and is meant for humans to read. Instead, strongly consider rewriting save.sh to copy file by file and in case of cp returning non-zero exit status, write the filename yourself in an "errors file" as a zero separated stream.

# save.sh
find .... -print0 |
while IFS= read -d '' -r file; do
     if ! cp "$file" "$dst"; then
          printf "%s\0" "$file" > errorsfile
     fi
done

# one rrors.sh
while IFS= read -d '' -r file; do
     echo "do something with $file"
done < errorsfile