Home > Mobile >  What is illegal about using curl with this URL?
What is illegal about using curl with this URL?

Time:10-06

I have the below URL:

https://amazon-advertising-api-reports-prod-euamazon.s3.amazonaws.com/apj6jra9ng5v4-avp9363n2w589-report-campaign-45f7c968-d7e8-4d8e-a6ef-2b144268ab36.json.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210930T164837Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=AKIXXXXXXXXXZ6X/20210930/us-east-1/s3/aws4_request&X-Amz-Signature=2a549XXXXXXXXXX0e61f43166c267eaafedf8

It's an original response from Amazon's advertising API with just a few elements replaced by 'X' where there are normal characters you wolud expect in a signature or Credential.

If I copy and paste this url into my browser I can download the file as I would expect. When trying to retrieve this file using any variation of the following curl commands ...

curl -o $REPORT_DATA_FILE_GZ "$REPORT_DOWNLOAD_URL\"

or 

curl -o $REPORT_DATA_FILE_GZ "\"$REPORT_DOWNLOAD_URL\""

or 

curl -o $REPORT_DATA_FILE_GZ $REPORT_DOWNLOAD_URL

... I am getting the error URL using bad/illegal format or missing URL. But I'm certain I am passing the REPORT_DOWNLOAD_URL and REPORT_DATA_FILE_GZ variables with correct values.

I display them with echo just before the call to curl to ensure they are valid.

I also cannot see any characters that look invalid to me in the URL.

All requests to curl are made before my test calls in the browser, so the short expiry period provided by the amazon-advertising-api (30 seconds) would not have expired by that time.

As per the comment I'm running this under Ubuntu 20.04, on WSL2, but most other curl request are working fine.

CodePudding user response:

I finally found the solution based on the output suggested by @dave_thompson_085. I extracted the $REPORT_DOWNLOAD_URL from the header of a different response and it still contained the \r at the end.

  • Related