Reducing amout of lines in variable within loop in Powershell-CodePudding

I have a txt file containing 10000 lines. Each line is an ID. Within every loop iteration I want to select 100 lines, put them in a special format and do something. I want to do this until the document is finished. The txt looks like this:

406232C1331283
4062321N022075
4062321H316457

Current approach:

$liste = get-content "C:\x\input.txt"

foreach ($item in $liste) {

azcopy copy $source $target --include-pattern "*$item*" --recursive=true

}

The system will go throug the TXT file and make a copy request for every name it finds in the TXT file. Now the system is able to handle like 300 search-patterns in one request. like

azcopy copy $source $target --include-pattern "*id1*;*id2*;*id3*"

How can I extract 300 items from the document at once, separate them with semicolon and embedd them in wildcard? I tried to pipe everyting in a variable and work with -skip. But it seems not easy to handle :(

CodePudding user response：

Use the -ReadCount parameter to Get-Content to send multiple lines down the pipeline:

Get-Content "C:\x\input.txt" -ReadCount 300 | ForEach-Object {

    $wildCards = ($_ | ForEach-Object { "*$_*" } -join ';'
    azcopy copy $source $target --include-pattern $wildCards --recursive=true

}

CodePudding user response：

Do you want 100 or 300 at a time? ;-) I'm not sure if I really got what the endgoal is but to slice a given amount of elements in chunks of a certain size you can use a for loop like this:

$liste = Get-Content -Path 'C:\x\input.txt'
for ($i = 0; $i -lt $Liste.Count; $i  = 100) {
    $Liste[$i..$($i   99)] 
}

Now if I got it right you want to join these 100 elements and surround them with certain cahrachters ... this might work:

'"*'   ($Liste[$i..$($i   99)] -join '*;*')   '*"'

Together it would be this:

$liste = Get-Content -Path 'C:\x\input.txt'
for ($i = 0; $i -lt $Liste.Count; $i  = 100) {
    '"*'   ($Liste[$i..$($i   99)] -join '*;*')   '*"'
}

CodePudding user response：

There's many ways, here's one of them...

First I would split array to chunks of 100 elements each, using this helper function:

Function Split-Array ($list, $count) {
    $aggregateList = @()

    $blocks = [Math]::Floor($list.Count / $count)
    $leftOver = $list.Count % $count
    for($i=0; $i -lt $blocks; $i  ) {
        $end = $count * ($i   1) - 1

        $aggregateList  = @(,$list[$start..$end])
        $start = $end   1
    }    
    if($leftOver -gt 0) {
        $aggregateList  = @(,$list[$start..($end $leftOver)])
    }

    $aggregateList    
}

For example to split your list into chunks of 100 do this:

$Splitted = Split-Array $liste -count 100

Then use foreach to iterate each chunk and join its elements for the pattern you need:

foreach ($chunk in $Splitted)
{
$Pattern = '"'   (($chunk | % {"*$_*"}) -join ";")   '"'
azcopy copy $source $target --include-pattern $Pattern --recursive=true
}