How to execute cURL requesting which requires to upload file in Golang?-CodePudding

I have a cURL request as follows.

$(curl --request PUT --upload-file "<path to catalog file on your local machine>" "<presigned URL>")

Let's say that I have to upload a bin/test.txt file with the presigned URL being https://www.someurl.com

I execute the command in my terminal curl --request PUT --upload-file "bin/test.txt" "https://www.someurl.com" and it works fine.

How do I write a piece of Golang which does the same? I have tried

cmd := exec.Command("curl", "--request", "PUT", "--upload-file", fmt.Sprintf("\"%s\"", catalogPath), fmt.Sprintf("\"%s\"", presignedURL))
err = cmd.Run()

but found no success.

CodePudding user response：

I see one obvious problem preventing that curl call from working properly, one quite possible and another one also possible.

The obvious problem is that string quoting — such as executing curl … --upload-file "bin/test.txt" … — in interpreted by the shell which is to execute the command. Quoting — using either double or single quotes — is used to inhibit interpreting of otherwise special characters by the shell; chiefly it's used to prevent the shell from splitting a string into separate "words" on whitespace characters or their series.
The key takeaway is that the command run by the shell after it's fully parsed the command to be executed (and interpreted the quotes) does not "see" these quotes because they are removed by the shell.

os/exec.Cmd calls the specified program directly and does not "pass it through" the shell. Hence if you include double quotes into the command-line parameters of the program to execute, they are passed to that program, unchanged. This means, curl were to try to find the file named test.txt" located in the directory named "bin — which is most probably not what you expected.
The same applied to the URL.

The second — possible — problem is that your call relies on the current directory of your Go program because you pass a relative path to curl.
This might or might not be a problem but you might check this anyway.

The third problem is that you might want to pass your URL through the "percent escaping" algorithm before passing it to curl.
You might look at PathEscape and QueryEscape functions of the net/url package.

Two pieces of advice follow.

First, I would try very hard not to call out to curl to perform such a ridiculously simple task. Go has excellent support for making HTTP requests (and serving them, FWIW) in its standard library, and PUTting a file is really a no-brainer with a solutions googleable in, like, five minutes.

Second, if, for some reason, you intend to stick with calling curl, consider passing it some options to make it fail loudly on errors — otherwise you're doomed to be in that „but found no success” situation in your attempts. For instance, consider passing curl the -s and -S command line options (together).

CodePudding user response：

that's not how you quote shell arguments, that would break if your argument starts with or ends with \ or ", the proper way to quote shell arguments on unix would be

func quoteshellarg(str string) string {
    if strings.Contains(str, "\x00") {
        panic("argument contains null bytes, it is impossible to escape null bytes in shell arguments!")
    }
    return "'"   strings.ReplaceAll(str, "'", "'\\''")   "'"
}

and with that, just

cmd := exec.Command("curl --request PUT --upload-file "   quoteshellarg(catalogPath)  " "   quoteshellarg(presignedURL));

... at least that's how to do it on unix systems. as for how to do it on Windows, it seems nobody knows for sure, not even Microsoft