In powershell, how can I retry a download after a certain amount of time if the file does not yet ex-CodePudding

I have a powershell script that downloads file like this:

powershell -Command `$progressPreference = 'silentlyContinue'; Invoke-WebRequest https://ftp.ncep.noaa.gov/data/nccf/com/cfs/prod/cfs/cfs.$m2/$m21z/6hrly_grib_04/pgbf$m2h.04.$m2h.grb2 -OutFile C:\OpenGrADS-2.2\data\cfs\cfs001.grb2`

Is it possible to check to see if the file exists before downloading, if it does, then download it, but if it doesn't, then wait a certain period of time before trying again.

CodePudding user response：

Spontaneously, I would say that the following function should accomplish everything you intend to do. Provided that the remote station sends a status code 404:

function Invoke-Download
{
    param (
        [Parameter()] [string] $uri,
        [Parameter()] [string] $outputPath,
        [Parameter()] [int] $sleeptimer = 30,
        [Parameter()] [int] $maxTries = 3,
        [Parameter()] [int] $tries = 0
    )

    try
    {
        Invoke-WebRequest -Uri $uri -OutFile $outputPath;
    }
    catch [System.Net.WebException]
    {
        $tries  ;

        if($tries -lt $maxTries)
        {
            Start-Sleep -Seconds $sleeptimer
            Invoke-Download -uri $uri -outputPath $outputPath -sleeptimer $sleeptimer -maxTries $maxTries -tries $tries
        }
        else
        {
            Write-Error ("Tried to download '{0}' {1}-times and failed" -f $uri, $tries);
        }
    }
}

Feel free to let me know if the answer is helpful to you.

CodePudding user response：

I tried the following based on Dave's response:

function Invoke-Download
{
    param (
        [Parameter()] [string] $uri = "https://ftp.ncep.noaa.gov/data/nccf/com/hrrr/prod/hrrr.$m2/conus/hrrr.t$m21z.wrfsfcf00.grib2.idx",
        [Parameter()] [string] $outputPath = "C:\OpenGrADS-2.2\data\hrrr\test.grb2",
        [Parameter()] [int] $sleeptimer = 30,
        [Parameter()] [int] $maxTries = 3,
        [Parameter()] [int] $tries = 0
    )

    try
    {
        Invoke-WebRequest -Uri $uri -OutFile $outputPath;
    }
    catch [System.Net.WebException]
    {
        $tries  ;

        if($tries -lt $maxTries)
        {
            Start-Sleep -Seconds $sleeptimer
            Invoke-Download -uri $uri -outputPath $outputPath -sleeptimer $sleeptimer -maxTries $maxTries -tries $tries
        }
        else
        {
            Write-Error ("Tried to download '{0}' {1}-times and failed" -f $uri, $tries);
        }
    }
}

I get this result...

C:\OpenGrADS-2.2\001-HRRR>powershell.exe -noexit -executionpolicy remotesigned -file "C:\OpenGrADS-2.2\001-HRRR\001-test.ps1"
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Try the new cross-platform PowerShell https://aka.ms/pscore6

PS C:\OpenGrADS-2.2\001-HRRR>