I am trying to setup hadoop cluster locally on three VMs. I have successfully setup my VMs. I have also installed hadoop on the master node. But, everytime I run vagrant provision, it downloads hadoop all over again which is time taking. I want to check if hadoop is already installed and skip the code that downloads hadoop if it is already installed.
Here is my code. /home/vagrant/hadoop is the location of hadoop.
# Download and extract Hadoop on master node (i.e node1)
if node.vm.hostname == "node1"
# Check if Hadoop is already installed so it wont install every time you run vagrant provision
if [ ! -d "/home/vagrant/hadoop" ]; then
node.vm.provision "shell", inline: <<-SHELL
wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.4/hadoop-3.3.4.tar.gz
tar -xvzf hadoop-3.3.4.tar.gz
mv hadoop-3.3.4 hadoop
SHELL
fi
fi
This is the error I am getting.
unexpected end-of-input, expecting
end'`
I changed the fi
to end
and I got the error below.
syntax error, unexpected
end', expecting end-of-input
CodePudding user response:
I think you're blurring the line between Ruby (Vagrant) and shell (host):
if node.vm.hostname == "node1"
# Check if Hadoop is already installed so it wont install every time you run vagrant provision
node.vm.provision "shell", inline: <<-SHELL
if [ ! -d "/home/vagrant/hadoop" ]; then
wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.4/hadoop-3.3.4.tar.gz
tar -xvzf hadoop-3.3.4.tar.gz
mv hadoop-3.3.4 hadoop
fi
SHELL
end
While the syntax of Ruby might, at least superficially, resemble the shell, it is quite different.
Remember, Ruby, running on your host machine, cannot test for directories that may or may not exist inside the VM. That's something you'll have to do with the shell, so it goes inside the shell command.