$ sign in url from text file gives error while download via wget-CodePudding

I want to download files with wget in a bash script. The url's look like this: https://xxx.blob.core.windows.net/somefolder/$somefile.webm?sv=xxx&sp=r..

Problem is the $ doller sign in the url

When I download the file with double quotes I get a 403 because the $ sign is probably interpreted.

wget "https://xxx.blob.core.windows.net/somefolder/$somefile.webm?sv=xxx&sp=r.."

When I single quote the url and download the file everything goes well:

wget 'https://xxx.blob.core.windows.net/somefolder/$somefile.webm?sv=xxx&sp=r..'

But the url should come from a line in a text file. So I read the file and pass the lines as url:

files=($(< file.txt))

# Read through the url.txt file and execute wget command for every filename
while IFS='=| ' read -r param uri; do 
    for file in "${files[@]}"; do
        wget "${file}"
    done
done < file.txt

I get the 403 here as well and don't know how to prevent the termimal from interpreting the dollar sign. How can I achieve this?

CodePudding user response：

But the url should come from a line in a text file.

If you have file with 1 URL per line or are able to easily alter your file to hold 1 URL per line, then you might use -i file option, from wget man page

-i file

--input-file=file

Read URLs from a local or external file. If - is specified as file, URLs are read from the standard input. (Use ./- to read from a file literally named -.)

If this function is used, no URLs need be present on the command line. If there are URLs both on the command line and in an input file, those on the command lines will be the first ones to be retrieved. If --force-html is not specified, then file should consist of a series of URLs, one per line.(...)

So if you have single file, say urls.txt you might use it like so

wget -i urls.txt

and if you have few files you might concat them and shove through standard input like so

cat urls1.txt urls2.txt urls3.txt | wget -i -

If file(s) contain additional data then remember to process them so GNU wget will get only URLs.