curl: (3) URL using bad/illegal format or missing URL - Checking response code-CodePudding

The code below works fine on Ubuntu 20.04. It checks the .csv file which contains URLs in column A. Every single address URL is in a new row.

To use it you need to run the script by typing:

bash script.sh file_with_urls.csv response_code

for example: bash script.sh urls-to-check.csv 200

#!/usr/bin/env bash
while read -r link; do
    response=$(curl --output /dev/null --write-out %{http_code} "$link")
    if [[ "$response" == "$2" ]]; then
        echo "$link"
    fi
done < "$1"

If I use it on Windows 10 with WSL Ubuntu 20.04 distribution I'm getting "curl: (3) URL using bad/illegal format or missing URL" error.

I'm a bit stuck with this...

CodePudding user response：

It's probably line endings:

#!/usr/bin/env bash

while IFS=, read -ra link; do
    response=$(curl --output /dev/null --write-out %{http_code} "${link[0]}")
    if [[ "$response" == "$2" ]]; then
       echo "${link[0]}"
    fi
done < <(sed 's/\r$//' "$1")

You can also do dos2unix urls_to_check.csv to convert it. If you open it in Windows, it could get converted back.

Alternatively, invoke it like this:

bash script.sh <(sed 's/\r$//' file_with_urls.csv) response_code