In Go, when running exec.Command with /usr/bin/script before a command, an error is thrown: /usr/bin-CodePudding

I am using Go to automate IBM Aspera uploads. I would like to capture the stdout/stderr percentage progress while this happens.

I am running this on an ubuntu:latest docker image

This is the entire command that I am trying to run in go.

/usr/bin/script -e -c 'ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>' > percentage.txt 2>&1

This is how I am calling this command in my Go project

cmd := exec.Command("/usr/bin/script", "-e", "-c", "'ascp", "-P <port>", "-m 3g", "-l 3g", "-i <secret>", "local_folder", "<user>@<host>:<remote-folder>'", "> percentage.txt", "2>&1")

I have found I have to use /usr/bin/script to capture the output of the ascp command, otherwise I am unable to capture the upload percentage from the stdout/stderr. I am only able to capture the success message and total bytes transferred.

Example percentage.txt output WITHOUT using /usr/bin/script:

Completed: 2389K bytes transferred in 1 seconds
 (10275K bits/sec), in 3 files, 1 directory.

Example percentage.txt output WITH using /usr/bin/script. As you can see, I am able to preserve the upload percentage:

Script started, output log file is 'typescript'.
<sample_file>                                                                                       100% 2194KB 13.3Mb/s    00:01    
<sample_file>                                                                                       100%  192KB 13.3Mb/s    00:01    
<sample_file>                                                                                       100% 3329   13.3Mb/s    00:01    
Completed: 2389K bytes transferred in 1 seconds
 (12268K bits/sec), in 3 files, 1 directory.
Script done.

When I take the raw command above, and run it directly on the cli of my docker instance, I have no issue and it works as expected.

However, when I attempt to run the command through the exec.Command() function, I receive this output

/usr/bin/script: invalid option -- 'P'
Try 'script --help' for more information.


exit status 1


panic: exit status 1

goroutine 1 [running]:
main.main()
        /project_dir/main.go:40  0x3e7
exit status 2

When I run println(cmd.String()) this is my output:

/usr/bin/script -e -c 'ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>' > percentage.txt 2>&1

And if I copy and paste the outputted command string from the cmd.string into my terminal, instead of running it through exec.Command(), it works and the percentage.txt captures the upload status.

What am I doing wrong here? Why is go's exec.Command() unable to run this, but my shell prompt is?

Full code:

func main() {
    f, err := os.OpenFile("percentage.txt", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
    if err != nil {
        panic(err)
    }
    defer f.Close()

    mwriter := io.MultiWriter(f, os.Stdout)

    err = os.Setenv("SHELL", "bash")
    if err != nil {
        return
    }
    cmd := exec.Command("/usr/bin/script", "-e", "-c", "'ascp", "-P <port>", "-m 3g", "-l 3g", "-i <secret>", "local_folder", "<user>@<host>:<remote-folder>'", "> percentage.txt", "2>&1")
    println(cmd.String())

    cmd.Stderr = mwriter
    cmd.Stdout = mwriter

    err = cmd.Run()
    if err != nil {
        println("\n\n"   err.Error()   "\n\n")
        panic(err)
    }
}

CodePudding user response：

You have mis-tokenized your command line. If this is the shell command:

/usr/bin/script -e -c 'ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>' > percentage.txt 2>&1

Then the corresponding exec.Command would almost be:

cmd := exec.Command("/usr/bin/script", "-e", "-c", "ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>")

That is, everything enclosed by single quotes on the command line is a single argument.

Using this version of the command, you will need to handle output redirection yourself (by reading the output from the command and writing it to a file).

If you really don't want to deal with reading the output from the command, you could explicitly call out to a shell:

cmd := exec.Command("/bin/sh", "-c", "/usr/bin/script -e -c 'ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>' > percentage.txt 2>&1")

...but this is typically less efficient and less robust, because now you need to be careful about characters (like >) that have special meaning to the shell and you need to be careful about properly quoting everything.