Remote execution of SSH command hangs in ruby using Net::SSH for a particular command-CodePudding

I found a similar question here, but the answer in such a question didn't work for me.

I am trying to connect the remote ssh server via ruby using Net::SSH.

It is working fine for me for all the commands provided via script and I could read the output of the command successfully.

But when I use the below command it is getting stuck in SSH.exec!(cmd) and control is not returned from the line. Only if i click Ctrl c in command line the script is getting ended.

sudo -S su root -c 'cockroach start --advertise-addr=34.207.235.139:26257 --certs-dir=/home/ubuntu/certs --store=node0015 --listen-addr=172.31.17.244:26257 --http-addr=172.31.17.244:8080 --join=34.207.235.139:26257 --background --max-sql-memory=.25 --cache=.25;'

This is the script I run from a SSH terminal with no issue:

sudo -S su root -c 'pkill cockroach'
sudo -S su root -c '
cd ~;
mv /home/ubuntu/certs /home/ubuntu/certs.back.back;
mkdir /home/ubuntu/certs;
mkdir -p /home/ubuntu/my-safe-directory;
cockroach cert create-ca --allow-ca-key-reuse --certs-dir=/home/ubuntu/certs --ca-key=/home/ubuntu/my-safe-directory/ca.key;
cockroach cert create-node localhost 34.207.235.139 172.31.17.244 $(hostname) --certs-dir /home/ubuntu/certs --ca-key /home/ubuntu/my-safe-directory/ca.key;
cockroach cert create-client root --certs-dir=/home/ubuntu/certs --ca-key=/home/ubuntu/my-safe-directory/ca.key;
        '
sudo -S su root -c 'cockroach start --advertise-addr=34.207.235.139:26257 --certs-dir=/home/ubuntu/certs --store=node0015 --listen-addr=172.31.17.244:26257 --http-addr=172.31.17.244:8080 --join=34.207.235.139:26257 --background --max-sql-memory=.25 --cache=.25;'

This is the Ruby script who attempts to do exactly the same, but it gets stuck:

require 'net/ssh'

ssh = Net::SSH.start('34.207.235.139', 'ubuntu', :keys => './plank.pem', :port => 22)

s = "sudo -S su root -c 'pkill cockroach'"
print "#{s}... "
puts ssh.exec!(s)

s = "sudo -S su root -c '
cd ~;
mv /home/ubuntu/certs /home/ubuntu/certs.back.#{rand(1000000)}};
mkdir /home/ubuntu/certs;
mkdir -p /home/ubuntu/my-safe-directory;
cockroach cert create-ca --allow-ca-key-reuse --certs-dir=/home/ubuntu/certs --ca-key=/home/ubuntu/my-safe-directory/ca.key;
cockroach cert create-node localhost 34.207.235.139 172.31.17.244 $(hostname) --certs-dir /home/ubuntu/certs --ca-key /home/ubuntu/my-safe-directory/ca.key;
cockroach cert create-client root --certs-dir=/home/ubuntu/certs --ca-key=/home/ubuntu/my-safe-directory/ca.key;
        '"
print "Installing SSL certifications... "
puts "done (#{ssh.exec!(s)})"

s = "sudo -S su root -c 'cockroach start --advertise-addr=34.207.235.139:26257 --certs-dir=/home/ubuntu/certs --store=node0015 --listen-addr=172.31.17.244:26257 --http-addr=172.31.17.244:8080 --join=34.207.235.139:26257 --background --max-sql-memory=.25 --cache=.25;'"
print "Running start command... "
puts "done (#{ssh.exec!(s)})"

# Use this command to verify the node is running:
# ps ax | grep cockroach | grep -v grep
s = "ps ax | grep cockroach | grep -v grep"
print "#{s}... "
sleep(10)
puts "done (#{ssh.exec!(s)})"

ssh.close

exit(0)

Here is the put put of the ruby script:

C:\code2\blackstack-deployer\examples>ruby start-crdb-environment.rb
sudo -S su root -c 'pkill cockroach'...
Installing SSL certifications... done ()
Running start command...

As you can see, the command gets stuck in the line Running start command...

I tried putting the command in the background:

s = "sudo -S su root -c 'cockroach start --advertise-addr=34.207.235.139:26257 --certs-dir=/home/ubuntu/certs --store=node0015 --listen-addr=172.31.17.244:26257 --http-addr=172.31.17.244:8080 --join=34.207.235.139:26257 --background --max-sql-memory=.25 --cache=.25 &'"
print "Running start command... "
puts "done (#{ssh.exec!(s)})"

but what happned is that the cockroach process never starts (the ps ax | grep cockroach | grep -v grep returns nothing)

CodePudding user response：

I figured out how to fix it.

I added > /dev/null 2>&1 at the end of the command, and it worked.

cockroach start --background --max-sql-memory=.25 --cache=.25 --advertise-addr=%net_remote_ip%:%crdb_database_port% --certs-dir=%crdb_database_certs_path%/certs --store=%name% --listen-addr=%eth0_ip%:%crdb_database_port% --http-addr=%eth0_ip%:%crdb_dashboard_port% --join=%net_remote_ip%:%crdb_database_port% > /dev/null 2>&1;

CodePudding user response：

Are there any logs/output in the cockroach-data/logs directory (should be located where you are running the start command from).

Or perhaps try redirecting stdout stderr to a file and seeing if there is any output there.

My hypothesis is that the CRDB process isn't starting correctly and so control isn't being returned to the terminal. The cockroachdb docs say that the --background flag only returns control when the crdb process is ready to accept connections. And the question/answer you linked noted that "SSH.exec! will block further execution until the command returns".