Vagrant machine wont run SED correctly-CodePudding

I have a problem I can't get around with vagrant provisioning. I have a file that has backslashes in it that I need to remove but Vagrant (Ruby?) just won't do it and I have tried for 2 days now.

Vagrant.configure("2") do |config|  
%w{test2 }.each_with_index do |name, i|
config.vm.define name do |node|
 node.vm.provider "virtualbox" do |vb|
   vb.name = "test#{i   2}"
   vb.memory = 512
   vb.cpus = 1
 end
 node.vm.box = "ubuntu/focal64"
 node.disksize.size = "1GB"
 node.vm.hostname = name
 node.vm.network :private_network, ip: "10.0.5.#{i   12}"
 node.vm.provision :shell, privileged: false, inline: <<-SHELL       
   cat /vagrant/control_join_file.sh | tr -d '\\n' > /vagrant/control_join_file#{i   12}.sh       
   sed -i -E "s@\\@ @" /vagrant/control_join_file#{i   12}.sh
   chmod  x /vagrant/control_join_file#{i   12}.sh 
   sudo /vagrant/control_join_file#{i   12}.sh 
 SHELL
end
end      
end

The line below is the culprit if I comment it out the provision happens.

sed -i -E "s@\\@ @" /vagrant/control_join_file#{i   12}.sh

I changed the delimiter to @ sign and have tried hundreds of things. I am really hoping someone can point out what's wrong here.

Also If I run it like this it works. I can look into the created file and sure enough the 4's are changed to percent signs. The issue seems to be with backslashes.

sed -i -E "s@4@%@" /vagrant/control_join_file#{i   12}.sh

CodePudding user response：

The issue turns out to be more complex than you would think. The problem is that you have three stages of string interpolation going on here. Each stage needs strings and backslashes escaped properly in order for the final sed command to be correct:

Ruby does string interpolation on the heredoc <<-SHELL string.
The shell under which the script is executed does its own interpolation.
Sed does its own interpolation on the regular expression you provide.

Let's work backwards starting from stage 3, so that we know what the end requirements are before we begin with the other stages.

Stage 3 - Sed

We know that sed requires the backslash to be escaped in its regular expression. So the sed command needs the following string s/\\/ /.

The question then becomes: how do we feed exactly that string to sed?

Stage 2 - The Shell

We KNOW we have to feed two backslashes to sed. Let's run some test in the shell and see how it escapes backslashes:

$ echo "\\"
\

Aha. It turns out the shell (bash) interpolates backslashes inside double quotes. What about single quotes?

$ echo '\\'
\\

Well that's useful. It means we can use single quotes to prevent backslash interpolation in Bash.

Let's test it:

$ echo '\\' | sed -E 's/\\/X/'
X\

It seems to work. Note that if you want to replace all backslashes, you need to use the g modifier:

$ echo '\\' | sed -E 's/\\/X/g'
XX

Ok. It seems we have gathered all the puzzle pieces for moving to stage 1.

Stage 1 - Ruby

In Ruby the heredoc <<-SHELL operator does its own string interpolation. Therefore backslashes need to be escaped.

Let's test it in irb:

> puts <<-SHELL
sed -E 's/\\\\/ /' ...
SHELL

sed -E 's/\\/ /' ...

Looks good!

What we learned so far:

Ruby heredoc <<-SHELL requires backslashes to be escaped.
The shell (Bash or sh), requires backslashes to be escaped, unless single quotes are used.
Sed needs two backslashes.

Now we can assemble our script properly:

node.vm.provision :shell, privileged: false, inline: <<-SHELL
  ...
  sed -i -E 's/\\\\/ /' /vagrant/control_join_file#{i   12}.sh
  ...
SHELL