Python subprocess not passing curly brackets as arguments? Or a problem with double quotes-CodePudding

I have a problem with } brackets and double quotes in the python subprocess module. this is an example of the standard terminal command:

curl -X POST localhost:8080/employees -H 'Content-type:application/json' -d '{"name": "Luke Skywalker", "role": "Jedi"}'

when I try to use the subprocess module like this: 1.

    test = subprocess.Popen(
        [
            "curl",
            "-X",
            "POST",
            "localhost:8080/employees",
            "-H",
            "'Content-type:application/json'",
            "-d",
            "'{",
            "name:",
            '"Luke Skywalker"',
            '"role:"',
            '"Jedi"',
            "}'",
        ],
        stdout=subprocess.PIPE,
    )
    output = test.communicate()[0]

I get:


  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   126    0   124  100     2   4225     68 --:--:-- --:--:-- --:--:--  5727
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (6) Could not resolve host: "Jedi"
curl: (3) unmatched close brace/bracket in URL position 1:
}'
 ^

Also with this:

    test = subprocess.Popen(
       [
            "curl",
            "-X",
            "POST",
            "localhost:8080/employees",
            "-H",
            "'Content-type:application/json'",
            "-d",
            '\'{"name":',
            '"Luke Skywalker"',
            '"role:"',
            ' "Jedi"}\' ',
        ],
        stdout=subprocess.PIPE,
    )
    output = test.communicate()[0]

I get:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   133    0   124  100     9   2699    195 --:--:-- --:--:-- --:--:--  3325
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) unmatched close brace/bracket in URL position 8:
 "Jedi"}' 
       ^

Any help would be greatly appreciated. Thanks in advance.

CodePudding user response：

You need to understand how the shell parses the command. When you execute:

curl -X POST localhost:8080/employees -H 'Content-type:application/json' -d '{"name": "Luke Skywalker", "role": "Jedi"}'

the arguments to curl are: -X, POST, localhost:8080/employees, -H, Content-type:application/json, -d, and {"name": "Luke Skywalker", "role": "Jedi"}

The point of the single quotes in the shell invocation is to keep the entire string as the literal argument and prevent it from being split on whitespace. Also, it is a mechanism for passing the double-quotes in the argument and prevent them from being treated as quote characters by the shell. The single quotes around Content-type:application/json are unnecessary since that string does not contain any characters that are special to the shell, but it is common practice to quote longish strings like that. For stylistic consistency, -H and POST and -X and even curl ought to also be quoted, but most adherents to the "quote everything if you don't understand the shell" style rarely adopt that level of consistency. curl never sees the single quotes, and attempting to pass them in the argument list from Python is a mistake. The initial curl command could also be called in the shell as:

'curl' "-X" PO""ST lo'cal'"host:8080/emp"loyees -"H" Content-type:application/json -d \{\"name\":\ \"Luke\ Skywalker\"\,\ \"role\":\ \"Jedi\"\}

Quoting in the shell is something that is (perhaps) worth studying a bit to understand. An important thing to understand is that the (unescaped) quote marks (double and single, but the rules are slightly different for each) are removed by the shell so that the called application never sees them.

Note that I was being a bit flippant above about quoting. It is perfectly reasonable to quote the string Content-type:application/json, since it is not worth the time deciding whether : and / are special to the shell. They are not, but there is enough ambiguity in shell implementations that it is very much worth quoting them. For instance, a posix compliant shell would execute the string cmd {a,b} by invoking cmd with the string {a,b}, but bash (even with --posix) will invoke cmd with the two arguments a and b. This is relevant here, since the json contains a , and that argument will be parsed by different shells differently if it is not quoted. When in doubt, use quotes. And always quote your variables unless there is a specific reason not to.

CodePudding user response：

Each item in the list is an argument to the process called. You can see that, you used 'Content-type:application/json' as a single argument. Similarly you can use the json as well.

See below:

payload = '{"name":"Luke Skywalker","role:"Jedi"}'
test = subprocess.Popen(
       [
            "curl",
            "-X",
            "POST",
            "localhost:8080/employees",
            "-H",
            "'Content-type:application/json'",
            "-d",
            f"'{payload}'"
        ],
        stdout=subprocess.PIPE,
    )
output = test.communicate()[0]

I made the data to a variable, so you can easily escape it.