Replacing a string with the value of a variable using shell script-CodePudding

I want to replace a string in a file with the value of a variable. I a string lvl in the template file prm.prm which needs to be replaced by the value of SLURM_ARRAY.

I tried using

sed -i 's/lvl/${SLURM_ARRAY}/' prm.prm

This replaces the string lvl with ${SLURM_ARRAY} and not its value. How can I rectify this?

CodePudding user response：

Every character between single quotes is used literally.

You could use double quotes instead as follows:

sed -i "s/lvl/${SLURM_ARRAY}/" prm.prm

However, your code now suffers from a code injection bug. There are characters (e.g. /) that will cause problems if found in the value of SLURM_ARRAY. To avoid this, these characters needs to be escaped.

quotemeta() { printf %s "$1" | sed 's/\([^a-zA-Z0-9]\)/\\\1/g'; }

sed -i "s/lvl/$( quotemeta "$SLURM_ARRAY" )/" prm.prm

It's best to avoid generating program from the shell. But that would require avoiding sed since it doesn't provide the necessary tools. For example, a couple of Perl solutions:

perl -i -spe's/lvl/$s/' -- -s="$SLURM_ARRAY" prm.prm

S="$SLURM_ARRAY" perl -i -pe's/lvl/$ENV{S}/' prm.prm

CodePudding user response：

Replace pattern with the value of an environment variable, with no extra interpolation of the contents:

Using perl:

export SLURM_ARRAY
perl -pe's/lvl/$ENV{SLURM_ARRAY}/g' prm.prm

Using awk:

export SLURM_ARRAY

awk '
{
    if (match($0, /lvl/)) {
        printf "%s", substr($0, 1, RSTART - 1)
        printf "%s", ENVIRON["SLURM_ARRAY"]
        print substr($0, RSTART   RLENGTH)
    }
    else {
        print
    }
}
' prm.prm

There's also SLURM_ARRAY=$SLURM_ARRAY perl ...etc or similar, to set the environment of a single process.

It can also be done with the variable as an argument. With both perl and awk you can access and modify the ARGV array. For awk you have to reset it so it's not processed as a file. The perl version looks like perl -e 'my $r = $ARGV[0]; while (<STDIN>) {s/lvl/$r/g; print}' "$SLURM_ARRAY" < prm.prm. It looks even better as perl -spe's/lvl/$r/g' -- -r="$SLURM_ARRAY". Thanks to ikegami.

For awk, I should say that the reason for not using awk -v r=foo is the expansion of C escapes. You could also read the value from a file (or bash process substitution).