curl long url with query parameters-CodePudding

I am trying to curl a long URL like the following and save it into a file:

https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ==.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603

where the filename should be only 277901819_162489612877754_7456098591865793913_n.jpg

curl "https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ==.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603" doesnt save it to a file

CodePudding user response：

You curl command is missing quote, it should be:

curl -o a.jpg 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ==.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'

CodePudding user response：

Okay, so you have tried the -o and -O options, but they don't seem to do what you want.

$ curl -o 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ==.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information

This fails because -o requires you to specify the filename yourself. The correct way to use -o is to specify the filename yourself:

$ curl -o '277901819_162489612877754_7456098591865793913_n.jpg' 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ==.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  175k  100  175k    0     0  85047      0  0:00:02  0:00:02 --:--:-- 85527

You haven't really explained why this isn't satisfactory, so I'll assume that you are getting the URL from user input or a file or something.

This is what the -O option is for. But it turns up another problem:

$ curl -O 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ==.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0Warning: Failed to create the file
Warning: 277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_n
Warning: c_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX
Warning: -prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyO
Warning: Q==.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&
Warning: oe=6266DFF7&_nc_sid=83d603: File name too long
  0  175k    0  1492    0     0   1406      0  0:02:07  0:00:01  0:02:06  1426
curl: (23) Failure writing output to destination

Depending on your system, perhaps this succeeded, but the filename wasn't what you wanted. This is trying to include the entire query string in the filename.

This seems about all you can do with curl as it stands. You either have to determine the filename yourself (such as with sed) and pass it with the -o option, or you need to use something other than curl.

I note that HTTPie handles this like you'd hope. Perhaps you could use it instead of curl?

$ http --download 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ==.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
HTTP/1.1 200 OK
Cache-Control: max-age=1209600, no-transform
Connection: keep-alive
Content-Length: 179351
Content-Type: image/jpeg
Date: Thu, 21 Apr 2022 12:25:45 GMT
Last-Modified: Wed, 06 Apr 2022 06:31:15 GMT
X-FB-Server-Cluster-Forwarded: syd2c01
X-Frame-Options: DENY
content-digest: adler32=3445021316
cross-origin-resource-policy: same-origin
timing-allow-origin: *
x-haystack-needlechecksum: 1987900843
x-needle-checksum: 1431545784

Downloading 175.15 kB to "277901819_162489612877754_7456098591865793913_n.jpg"
Done. 175.15 kB in 1.08534s (161.38 kB/s)