Run a function in parallel and wait for output-CodePudding

Is it possible to call a powershell function or script several times in parallel and wait for the data to return before moving on? I have done this in Linux and my pseudo code is a more like how it's done in linux as an example.

Say we have a function called 'Get-OSRebootDates' which returns the recent reboots as a result string. Instead of running it serially, can I run it in parallel?

Normal serial method

$Result = @()
ForEach ($pcname in $pclistarray) {
     $Result  = Get-OSRebootDates -Target $pcname
     # call it one at a time and wait before moving on to the next  
} 
# then do something with data

Pseudo code of what I would like to do in parallel:

$Result = @()
$($Result  = Get-OSRebootDates -Target $pcname1)  &
$($Result  = Get-OSRebootDates -Target $pcname2)  &
$($Result  = Get-OSRebootDates -Target $pcname3)  &
$($Result  = Get-OSRebootDates -Target $pcname4)  &
Wait 
# then do something with the data

FYI example function

Function Get-OSRebootDates {
# Summary: Search Application log for recent reboots in hours (e.g. $RangeNegHours -> -340) (this search is fast)
# Params: Target is howt, Range is how far to look back 
Param(
    [Parameter(Mandatory = $true)]$Target, 
    [Parameter(Mandatory = $false)]$RangeNegHours = -8760 # default 1 year 
)
$filters = @{}
$filters.Add("StartTime", ((Get-Date).AddHours([int]$RangeNegHours)))
$filters.Add("EndTime", (Get-Date))
$filters.Add("LogName", "System")
$filters.Add("ID", 6005) #6005 started, #6006 shutdown
$filters.Add("providername", '*EventLog*')
$filters.Add("level", 4)
$RebootDates = [string]::Empty
Try { 
    $LogRaw = Get-WinEvent -ErrorAction Stop -FilterHashtable $filters -MaxEvents 2500000  -ComputerName $Target | select-object id, machinename, timecreated, message, LevelDisplayName, ProviderName
}
Catch {
    Write-Host ("Something went wrong with ["   $MyInvocation.MyCommand   "] for "   $Target   " ..") -ForegroundColor Red
    start-sleep -seconds 4
    return $RebootDates
}

$Count = 0
ForEach ($Item in $LogRaw) {
    If ([string]$Item.TimeCreated) {
        $RebootDates = $RebootDates   [string]$Item.TimeCreated   "`n"
        $Count  = 1
        If ($Count -gt 5) {
            break;
        }
    }
}
Return [string]$RebootDates

}

CodePudding user response：

tl;dr regarding your Normal serial method, using = on a System.Array is highly inefficient, since arrays are collections of a fixed size, PowerShell has to recreate it on each iteration of your foreach loops. See this Q&A for a much better explanation.

This should be your go to Normal serial method, or you can use System.Collections.Generic.List<T> or System.Collections.ArrayList.

$Result = ForEach ($pcname in $pclistarray) {
     Get-OSRebootDates -Target $pcname
     # call it one at a time and wait before moving on to the next  
}

Regarding your main question, How to run a function in parallel and wait for output, first of all, in my personal experience when working with Get-WinEvent I have seen much better (faster) results doing this:

Invoke-Command -ComputerName server01 -ScriptBlock { Get-WinEvent .... }

Than doing this:

Get-WinEvent .... -ComputerName server01

In addition, Invoke-Command can handle very well the multithreading for you since -ComputerName accepts a string[] (an array with hosts).

Invoke-Command [[-ComputerName] <String[]>]
Get-WinEvent [-ComputerName <String>]

Here goes the basic example of you can, first load the function in the Job's scope and then invoke it. Note, this is running 11 Jobs in parallel. If you're a looking for a more "efficient multithreading" since Start-Job is arguably slower than a normal linear loop consider using the TheadJob module or use Runspace.

function sayhello ([string]$Name, [int]$Job) {
    "Hello $Name from Job $Job..."
    Start-Sleep 10
}

$funcDef = "function sayhello {$function:sayhello}"

# Each Job should take 10 seconds
$elapsed = [System.Diagnostics.Stopwatch]::StartNew()

0..10 | ForEach-Object {
    Start-Job -ScriptBlock {
        param($def, $name, $job)

        . ([scriptblock]::Create($def)) # => Load the function in this scope
        sayhello -Name $name -Job $job

    } -ArgumentList $funcDef, 'world', $_
} | Receive-Job -AutoRemoveJob -Wait

$elapsed.Stop()
"`nElapsed Time: {0} seconds" -f
$elapsed.Elapsed.TotalSeconds.ToString('0.##')

Result:

Hello world from Job 1...
Hello world from Job 6...
Hello world from Job 0...
Hello world from Job 4...
Hello world from Job 3...
Hello world from Job 5...
Hello world from Job 2...
Hello world from Job 10...
Hello world from Job 7...
Hello world from Job 9...
Hello world from Job 8...

Elapsed Time: 13.82 seconds