In the example below, even though I cap the concurrent threads at 50, the code will (within seconds) process and go down to the foreach ($thread in $threads).

Is this expected behavior because the runspace pool is doing wizardry in the background and will store all 2000 thread objects in memory, but only allow 50 active at any time?

The reason behind my question is that I'm trying to figure out why the processing is not nearly as fast as it should be.

$runspacePool = [runspacefactory]::CreateRunspacePool(

    1, #Min Runspaces

    50 #Max Runspaces

);

$remoteMailboxes = Get-RemoteMailbox -Resultsize 2000;

foreach ($remoteMailbox in $remoteMailboxes)
{
    $powerShell = [powershell]::Create($sessionState);
    $powerShell.RunspacePool = $runspacePool;
    
    [void]$powerShell.AddScript({   
        Param ($alias<#, $paths, $filters, $exclusionsArrNoAsterisks#>)     
        [pscustomobject]@{

            alias = $alias;

        } | Out-Null;
        
# a bunch of processing that takes 30 seconds happens here  
        
        return $returnedMailboxObj;

    }) # end of add script

    $powerShell.AddParameter('alias', $remoteMailbox.Alias) | Out-Null;
    $returnVal = $powerShell.BeginInvoke();
    $temp = "" | Select powerShell,returnVal,server;
    $temp.powerShell = $powerShell;
    $temp.returnVal = $returnVal;
    $temp.server = $server;
    $threadStartTime = [DateTime]::Now.ToLongTimeString();
    $tuple = [tuple]::create([string]$server, [DateTime]$threadStartTime); 
    $threadTimerTemp.Add($tuple) | Out-Null;
    $threads.Add($temp) | Out-Null;
}

foreach ($thread in $threads)
{
    $threadsTemp.Add($thread) | Out-Null;
}

$endInvokeArr = New-Object System.Collections.ArrayList;
$threadsCompleted = New-Object System.Collections.ArrayList;
$threadsNotCompleted = New-Object System.Collections.ArrayList;
$threadsNotCompleted.Add("PlaceHolder") | Out-Null;

while ($threadsTemp.Count -gt 0)
{
    $threadsNotCompleted.Clear();
    Write-Host "Updated thread count" $threadsTemp.Count;
    for ($i = 0; $i -lt $threadsTemp.Count; $i  )
    {
        $threadIsCompleted = $threadsTemp[$i].returnVal.IsCompleted;
        if ($threadIsCompleted -eq $false)
        {
            #$completed = $false;
            $threadsNotCompleted.Add($thread) | Out-Null;
            # Ignore
        }
        else
        {
            $threadHandle = $threadsTemp[$i].returnVal.AsyncWaitHandle.Handle;
            $threadsCompleted.Add($threadHandle) | Out-Null;
            $endInvoke = $threadsTemp[$i].PowerShell.EndInvoke($threadsTemp[$i].returnVal);
            $endInvokeArr.Add($endInvoke) | Out-Null;
            $threadsTemp.Remove($threadsTemp[$i]);
            $i  ;
        }
    }
    
    Write-Host "endInvokeArrCount" $endInvokeArr.Count;
    Write-Host "Threads completed count: " $threadsCompleted.Count;
    Write-Host "Threads count" $threadsTemp.Count;      

    sleep -Milliseconds 100;
} # while end   

CodePudding user response：

Is this expected behavior because the runspace pool is doing wizardry in the background and will store all 2000 thread objects in memory, but only allow 50 active at any time?

Yes! This is the expected behavior

The method PowerShell.BeginInvoke() could as well have been called PowerShell.ScheduleWorkForWheneverARunspaceIsReadyInTheFuture() - that is, BeginInvoke doesn't actual do any work before returning to the caller - it simply:

• Ensures that an available runspace or runspacepool exists,
• Sets up the "pipes and duct-tape" needed for collecting output from the code to be run
• Adds a "work order" to a queue to "schedule" that some runspace picks up the request
• Returns a "receipt" (the IAsyncResult object you briefly store in $returnval) for the "work order" scheduled

These relatively simple steps can obviously be completed much faster than actually executing the associated code, so it's not actually that unusual to be able to begin (ie "schedule") many more executions than can be run concurrently - heck, the runspace pool might not even have started executing the first request by the time you've scheduled the remaining 1999.