Home > Back-end >  Why running curl command using parallel needs to timeout?
Why running curl command using parallel needs to timeout?

Time:04-09

I have set up a daphne server with my Django application and I wanted to use the curl command to send parallel requests to the API that I have created. This is the command that I am using in a terminal.

seq 10 | parallel -j 5 --joblog pandu.log curl --connect-timeout 2 -v -F "FILE=@PATH_TO_FILE" http://127.0.0.1:8000/<wxy>/<xyz>

So, I started the daphne server in one terminal and ran the curl command in another terminal. This is the view that I have created for handling API requests.

@csrf_exempt
def inputView(request):
    file = request.FILES['FILE']
    call_print_primers(file)
    return HttpResponse("well I have recieved the file.")

This is the output I am getting when I ran the curl command.

Trying 127.0.0.1:8000...

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
> POST /<wxy>/<xyz> HTTP/1.1
> Host: 127.0.0.1:8000
> User-Agent: curl/7.82.0
> Accept: */*
> Content-Length: 2828
> Content-Type: multipart/form-data; boundary=------------------------4e8f33ef7353b32d
> 
} [2828 bytes data]
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< X-Frame-Options: DENY
< Content-Length: 63
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< 
{ [63 bytes data]
100  2891  100    63  100  2828     72   3275 --:--:-- --:--:-- --:--:--  3349
* Connection #0 to host 127.0.0.1 left intact
*   Trying 0.0.0.3:80...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0* After 2000ms connect time, move on!
* connect to 0.0.0.3 port 80 failed: Connection timed out
* Connection timeout after 2000 ms
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
* Closing connection 1
curl: (28) Connection timeout after 2000 ms
*   Trying 127.0.0.1:8000...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
> POST /<wxy>/<xyz> HTTP/1.1
> Host: 127.0.0.1:8000
> User-Agent: curl/7.82.0
> Accept: */*
> Content-Length: 2828
> Content-Type: multipart/form-data; boundary=------------------------4e31c748940a7fe5
> 
} [2828 bytes data]
* We are completely uploaded and fine
100  2828    0     0  100  2828      0  14076 --:--:-- --:--:-- --:--:-- 14069* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< X-Frame-Options: DENY
< Content-Length: 63
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< 
{ [63 bytes data]
100  2891  100    63  100  2828     72   3258 --:--:-- --:--:-- --:--:--  3330
* Connection #0 to host 127.0.0.1 left intact
*   Trying 0.0.0.1:80...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0* After 2000ms connect time, move on!
* connect to 0.0.0.1 port 80 failed: Connection timed out
* Connection timeout after 2000 ms
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
* Closing connection 1
curl: (28) Connection timeout after 2000 ms
*   Trying 127.0.0.1:8000...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
> POST /<wxy>/<xyz> HTTP/1.1
> Host: 127.0.0.1:8000
> User-Agent: curl/7.82.0
> Accept: */*
> Content-Length: 2828
> Content-Type: multipart/form-data; boundary=------------------------7845802a0524217a
> 
} [2828 bytes data]
* We are completely uploaded and fine
100  2828    0     0  100  2828      0  14074 --:--:-- --:--:-- --:--:-- 14069* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< X-Frame-Options: DENY
< Content-Length: 63
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< 
{ [63 bytes data]
100  2891  100    63  100  2828     72   3248 --:--:-- --:--:-- --:--:--  3319
* Connection #0 to host 127.0.0.1 left intact
*   Trying 0.0.0.2:80...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0* After 2000ms connect time, move on!
* connect to 0.0.0.2 port 80 failed: Connection timed out
* Connection timeout after 2000 ms
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
* Closing connection 1
curl: (28) Connection timeout after 2000 ms
*   Trying 127.0.0.1:8000...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
> POST /<wxy>/<xyz> HTTP/1.1
> Host: 127.0.0.1:8000
> User-Agent: curl/7.82.0
> Accept: */*
> Content-Length: 2828
> Content-Type: multipart/form-data; boundary=------------------------d3cacacac364da47
> 
} [2828 bytes data]
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< X-Frame-Options: DENY
< Content-Length: 63
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< 
{ [63 bytes data]
100  2891  100    63  100  2828     73   3290 --:--:-- --:--:-- --:--:--  3365
* Connection #0 to host 127.0.0.1 left intact
*   Trying 0.0.0.4:80...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0* After 2000ms connect time, move on!
* connect to 0.0.0.4 port 80 failed: Connection timed out
* Connection timeout after 2000 ms
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
* Closing connection 1
curl: (28) Connection timeout after 2000 ms

I am not getting why it is trying different IPs and why the connection is getting timed out.

This is the output of the terminal where the server is running.

2022-03-30 11:50:52,832 INFO     Starting server at tcp:port=8000:interface=127.0.0.1
2022-03-30 11:50:52,832 INFO     HTTP/2 support enabled
2022-03-30 11:50:52,832 INFO     Configuring endpoint tcp:port=8000:interface=127.0.0.1
2022-03-30 11:50:52,833 INFO     Listening on TCP address 127.0.0.1:8000
127.0.0.1:47994 - - [30/Mar/2022:11:50:57] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:47998 - - [30/Mar/2022:11:50:57] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:47992 - - [30/Mar/2022:11:50:57] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:47990 - - [30/Mar/2022:11:50:57] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:47996 - - [30/Mar/2022:11:50:57] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:48004 - - [30/Mar/2022:11:51:00] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:48006 - - [30/Mar/2022:11:51:00] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:48008 - - [30/Mar/2022:11:51:00] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:48002 - - [30/Mar/2022:11:51:00] "POST /<wxy>/<xyz>" 200 63
127.0.0.1:48000 - - [30/Mar/2022:11:51:00] "POST /<wxy>/<xyz>" 200 63
               

CodePudding user response:

I am not using parallel anymore, instead I am now using xargs. So, now the command would be

seq 100 | xargs -I $ -P 100 curl -v -F "FILE=@FILE_PATH" http://127.0.0.1:8000/xyz/wxy -b cookies.txt

  • Related