Problem with Quotes while using variables with "&"-CodePudding

I've been sitting here for 3 days now trying to solve the following problem. I need a call function that I can pass a url as a parameter to. However, this URL has an &, which no longer works as soon as I pass the variable as a call parameter. I can't set /& in the Url, because its a dynamic Url.

Every url begins with https:// and ends with &dl=1

@echo off
setlocal enabledelayedexpansion

set "url=!array[2]!"

call :do_download
   
:do_download 
    powershell -c "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; 
       Invoke-WebRequest '%url%' -OutFile 'C:\Program Files (x86)\test.zip'"

But if i try in all versions of quotes, doublequotes, with trimm or other tricks, it dont work :´( I have rly no idea yet. But i need the Url as call-param.

@echo off
setlocal enabledelayedexpansion

set "url=!array[2]!"

call :do_download "%url%"

:do_download 
    powershell -c "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; 
       Invoke-WebRequest '%1' -OutFile 'C:\Program Files (x86)\test.zip'"

If you try it self: you can save a packs.txt with:

Testlink
https://sync.luckycloud.de/d/76ffff3d76ea4e9ab15e/files/?p=/Test.txt&dl=1

The link is to download a .txt file with 200 words from lorem ipsum. and here is the complete code, that will work at this moment, but without call-param. you can set the paths whatever you want, it's only for example. This code runs by itself; the ! are used because later it will be inside an if, but that code is independent of this one.

@echo off
setlocal enabledelayedexpansion

call :readtest
pause
call :do_download
pause

:readtest
    set count=0
    for /f "usebackq tokens=*" %%A in ("C:\Program Files (x86)\Steam\steamapps\common\packs.txt") do (
        set /a count =1
        set url[!count!]=%%A
    )
    set "pack_sd=!url[2]!"
    goto :eof

:do_download 
powershell -c "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; Invoke-WebRequest '%pack_sd%' -OutFile 'C:\Program Files (x86)\Steam\steamapps\common\test.txt'"
goto :eof

If i try the answer with use: :"=

call :do_download "%pack_sd%"
    
:do_download 
    echo TT %1 TT
    set "url=%1"
    set test=%url:"=%
    echo "TT %test% TT"

first echo:

TT "https://sync.luckycloud.de/d/76ffff3d76ea4e9ab15e/files/?p=FTest.txt&dl=1" TT

second echo:

error: command "dl" is wrong or unknown...
"TT "= TT"

If i try the answer with %~1

call :do_download "%pack_sd%"

:do_download 
    echo TT %1 TT
    set url=%~1
    echo "TT %url% TT%

first echo:

TT "https://sync.luckycloud.de/d/76ffff3d76ea4e9ab15e/files/?p=FTest.txt&dl=1" TT

second echo:

error: command "dl" is wrong or unknown...
"TT https://sync.luckycloud.de/d/76ffff3d76ea4e9ab15e/files/?p=FTest.txt TT"

if i use the answer with: '%1:"=' inside powershell command:

call :do_download "%pack_sd%"

:do_download 
        powershell -c "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; Invoke-WebRequest '%1:"=' -OutFile 'C:\Program Files (x86)\Steam\steamapps\common\test.txt'"
    goto :eof 

error: "The string has no terminator: '."

CodePudding user response：

Update: Based on the latest edit to your question with ever-changing requirements, the problem boils down to this:

Use set "url=%~1" in your :do_download subroutine. This stores the unquoted value of %1 (the first parameter) in variable %url%
- Do not use set "url=%1" or set url=%~1, both of which predictably break with a %1 value that has embedded surrounding double quotes and contains &.
- Do not use %1:"=, which doesn't do what you think it does. %1 is a parameter, whereas the string-substitution syntax you're trying to use only works with variables (e.g., %url:"=%)
Then you can use %url% as-is inside the "..." string that forms the powershell -c argument (but, as shown in the next section, you could just use %~1 directly).

Here's a minimal example that shows that a URL containing & is correctly passed through to PowerShell; everything else in your code is incidental (use of for /f, delayed variable expansion, specifics of the PowerShell command):

@echo off & setlocal

call :do_download "http://example.org&dl=1"

goto :eof

:do_download
  set "url=%~1"
  powershell -c "Write-Output '%url%'"
goto :eof

The above prints verbatim http://example.org&dl=1, proving that the URL was correctly passed through to PowerShell.
Note that, since the unquoted URL is stored in intermediate variable %url%, you could alternatively take advantage of the fact that all cmd.exe variables (but not parameters such as %1) are also environment variables, which therefore allows PowerShell to access the URL via an environment-variable reference instead of embedding it directly in the command string (the equivalent of cmd.exe's %url% is $env:url in PowerShell):
- powershell -c "Write-Output $env:url"

Original answer, with some additional background information:

It's not clear where !array[2]! comes from, but here's a simplified example that works if the target URL is passed as the first argument (%1) to your batch file:

@echo off
setlocal

call :do_download %1

:: Exit here, so that the code below isn't executed again.
exit /b 

:do_download 
  powershell -c "Write-Output '%~1'"

The above, if invoked with, say, foo.cmd "http://example.org&more" from cmd.exe, would print verbatim http://example.org&more, proving that the value was correctly passed through to PowerShell.

%1, if its value contains &, must by definition have been passed in double quotes, otherwise the batch-file call itself would have broken.
Because of that, it is safe to pass it on as-is (unquoted) in the call statement.
However, in the context of the powershell call, the surrounding double quotes in the %1 value must be removed, so as not to interfere with the "..." surrounding the entire -c argument - that's what %~1 does.

If the %url% value is obtained from a file, as you state, only a slight tweak is necessary: because your %url% value then does not include embedded surrounding double quotes, pass it as "%url%":

@echo off
setlocal

:: Set %array[2]% to a sample value.
set "array[2]=http://example.org&more"

:: Reference it via delayed expansion (why?)...
set "url=!array[2]!"

:: ... and pass it double-quoted.
call :do_download "%url%"

:: Exit here, so that the code below isn't executed again.
exit /b 

:do_download 
  powershell -c "Write-Output '%~1'"

CodePudding user response：

Does this work?

@echo off
setlocal enabledelayedexpansion

set "url=!array[2]!"

call :do_download
   
:do_download 
    powershell -c "$Url=$Env:url; [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; 
       Invoke-WebRequest $Url -OutFile 'C:\Program Files (x86)\test.zip'"

You have Delayed Explansion enabled, so you should be able to just past the variable's name and expand it later.

Does this also work?

@echo off
setlocal enabledelayedexpansion

set "url=!array[2]!"

call :do_download url
GOTO :EOF

:do_download 
    powershell -c "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; 
       Invoke-WebRequest $Env:!%~1! -OutFile 'C:\Program Files (x86)\test.zip'"

EDIT:

I'm not sure if the packs.txt file contains only URLs or other content. I placed 5 copies of the sync.luckycloud.de URL, the one mentioned in the question, in a mock packs.txt file and was able to get this code to work.

@ECHO OFF
SETLOCAL EnableDelayedExpansion
GOTO :Start

:ReadTest
    SET Count=0
    FOR /F "USEBACKQ EOL=` TOKENS=* DELIMS=`" %%U IN ("C:\Program Files (x86)\Steam\steamapps\common\packs.txt") DO (
        SET /A Count =1
        SET url[!count!]=%%U
    )
    SET "pack_sd=!url[2]!"
GOTO :EOF

:Do_Download
    PowerShell -c "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; Invoke-WebRequest $Env:pack_sd -OutFile 'C:\Program Files (x86)\Steam\steamapps\common\test.txt'"
GOTO :EOF

:Start
    CALL :ReadTest
    CALL :Do_Download
GOTO :EOF

NOTE: I believe the characters below are not allowed as part of a URL. The FOR command first looks for DELIMS, but there should never be a back tick in the URL, so DELIMS should never be found. But if one is found, it is grabbed by DELIMS before EOL, this make EOL disabled. The default for EOL is a semicolon, which appears to be a valid character in URLs. So it is important to disable EOL, or give it a value that will never be in a URL.

"%\^`{|}<>

Final Generic Version:

ReadFile: Takes name of array for saving URLs and the path of a file containing URLs. Reads file, storing URLs in array.

Do_Downloads: Takes name of array of URLs and the path where files will be saved. URLs from array are downloaded and saved to files with name WebFile{N}.txt, where {N} is the line number the URL came from in the file containing URLs.

@ECHO OFF
SETLOCAL EnableDelayedExpansion
GOTO :Start

:ReadFile
    SET Count=0
    FOR /F "USEBACKQ EOL=` TOKENS=* DELIMS=`" %%U IN ("%~2") DO (
        SET /A Count =1
        SET %~1[!Count!]=%%U
    )
GOTO :EOF

:Do_Downloads
    FOR /L %%I IN (1, 1, %Count%) DO (
        SET Url=!%~1[%%I]!
        PowerShell -c "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; Invoke-WebRequest $Env:Url -OutFile '%~2\WebFile%%I.txt'"
    )
GOTO :EOF

:Start
    CALL :ReadFile Urls "C:\Program Files (x86)\Steam\steamapps\common\packs.txt"
    CALL :Do_Downloads Urls "C:\Program Files (x86)\Steam\steamapps\common"
GOTO :EOF