awk command in Perl's system does not work-CodePudding

I am writing a small Perl script that executes an Awk command :

I try to swap two columns in a file, the file is like this :

domain1,ip1
domain2,ip2
domain3,ip3

the result should be

ip1,domain1
ip2,domain2
ip3,domain3

The Perl command invoking awk is like this:

system("ssh -p 22 root\@$mainip 'awk -F, '{print $2,$1}' OFS=, /root/archive/ipdomain.txt > /root/ipdom.txt'");

This is the error I get :

awk: cmd. line:1: {print
awk: cmd. line:1:       ^ unexpected newline or end of string

any suggestions, please?

CodePudding user response：

I am using a shortened example here

system("ssh localhost 'awk '{print $2,$1}' file.txt'")

system() sees:
    ssh localhost 'awk '{print $2,$1}' file.txt'
local shell expands:
    ssh
    localhost
    awk 
    {print
    $2,$1}
     file.txt
local shell replaces $1 and $2 (positional args) with empty strings:
    ssh
    localhost
    awk 
    {print
    ,}
     file.txt
ssh executes:
    ssh localhost awk {print ,} file.txt
remote shell gets:
    awk
    {print
    ,}
    file.txt

So the remote shell runs awk with {print as its program argument, resulting in the described error. To prevent this, the invocation of system() can be changed to;

system("ssh localhost \"awk '{print \$2,\$1}' file.txt\"")

system() sees:
    ssh localhost "awk '{print \$2,\$1}' file.txt"
local shell expands:
    ssh
    localhost
    awk '{print \$2,\$1}' file.txt
ssh executes
    ssh localhost awk '{print \$2,\$1}' file.txt
remote shell gets
    awk
    {print \$2,\$1}
    file.txt
remote shell expands \ escapes
    awk
    {print $2,$1}
    file.txt

Remote awk now gets {print $2,$1} as its program argument, and executes successfully.

CodePudding user response：

With all the multi-level quoting and escaping that need be done right, no wonder it fails. Now, a complex command like that^† will always be tricky to run, but libraries help.

A properly quoted string to run through a shell can be formed with String::ShellQuote ^‡

use warnings;
use strict;
use feature 'say';

use String::ShellQuote qw(shell_quote);

die "Usage: $0 file outfile\n" if @ARGV != 2;

my ($file, $out) = @ARGV;

my @cmd_words = 
    ( 'ssh', 'hostname', 'awk', q('{print $2 $1}'), $file, '>', $out );

my $cmd = shell_quote @cmd_words;

system($cmd);

This swaps the first two words on each line and prints just those. It works as expected in my tests. Please adjust (the ssh, awk, and filenames) as needed.

The next helpful adjustment would be to use a library for ssh as well, like Net::OpenSSH.

The makeVoiceBot answer is informative and it got half way there but I find the need for

system("ssh hostname \"awk '{print \\\$2 \\\$1}' $path\"");

This works in my tests (on systems I ssh to).

^† This is a shell command which runs ssh, and then executes a command on the remote system which runs a shell (there) as well, in order to run awk and redirect its output to a file.

A bit more than an "awk command" as the title says.

^‡ The library can prepare a command for bash (as of this writing), but one can look at the source for it and adjust it for their own shell, at least. There is also Win32::ShellQuote