Is there a better way to write this to combine files-CodePudding

I am new to PowerShell, but i am slowly getting the hang of it.

I was wondering if there is a better way to write this? In a single directory I have monthly reports for 14 names as text files. The below looks at the directory, searches for the NAME1 and for any files containing Jan, Feb, Mar and combines them into a single file and exports the combined file to another location with a specific name:

Get-childitem -Path 'C:\Powershell\Attempt\*.txt' | where-object {$_.name -like "*NAME1**JAN*" -or $_.name -like "*NAME1**FEB*" -or $_.name -like "*NAME1**MAR*"} | get-content | Out-File C:\Powershell\Attempt\test\NAME1_Q1_.txt -Encoding ascii 
Get-childitem -Path 'C:\Powershell\Attempt\*.txt' | where-object {$_.name -like "*NAME1**APR*" -or $_.name -like "*NAME1**MAY*" -or $_.name -like "*NAME1**JUN*"} | get-content | Out-File C:\Powershell\Attempt\test\NAME1_Q2_.txt -Encoding ascii
Get-childitem -Path 'C:\Powershell\Attempt\*.txt' | where-object {$_.name -like "*NAME1**JUL*" -or $_.name -like "*NAME1**AUG*" -or $_.name -like "*NAME1**SEP*"} | get-content | Out-File C:\Powershell\Attempt\test\NAME1_Q3_.txt -Encoding ascii
Get-childitem -Path 'C:\Powershell\Attempt\*.txt' | where-object {$_.name -like "*NAME1**OCT*" -or $_.name -like "*NAME1**NOV*" -or $_.name -like "*NAME1**DEC*"} | get-content | Out-File C:\Powershell\Attempt\test\NAME1_Q4_.txt -Encoding ascii

Is this possible with a loop? or is it better to just write out all 14 names and quarterly combinations?

The above also creates blank Qx.txt files if the source files don't exist (does that make any sense) so I have also written this to remove those that are blank

get-childitem C:\powershell\attempt\test -Recurse | foreach {
   if($_.Length -eq 0){
      Write-Output "Removing Empty File $($_.FullName)"
      $_.FullName | Remove-Item -Force
   }
   }
   if( $_.psiscontainer -eq $true){
      if((gci $_.FullName) -eq $null){
         Write-Output "Removing Empty folder $($_.FullName)"
         $_.FullName | Remove-Item -Force
      }
}

Is there a way to incorporate this into the main script, or is it better to keep this as i "tidy up" at the end?

I do have another query, but I'm not sure if it's better being a separate post (I don't want to put too much in this one if it's not the way) It is about how to rename the files from different variables. I can get the different name variables, but not working harmoniously within the above script - this comes down to my lack of knowledge

Many thanks in advance, Kind Regards

CodePudding user response：

Before trying to create a loop, let's do something about those clunky -like clauses. We can make them go away with a single -match clause.

The expression

$_.name -like "*NAME1**JAN*" -or $_.name -like "*NAME1**FEB*" -or $_.name -like "*NAME1**MAR*"

is equivalent to

$_.name -match "NAME1.*(JAN|FEB|MAR)"

The -like operator uses Wildcards. Wildcards are nice, but the -match operator works with full fledged Regular Expressions which are much more versatile.
These are not compatible with each other - some Wildcard expressions are valid Regular Expressions and vice versa, but match different strings.
Don't forget that -match and -like are not case-sensitive. For case-sensitive comparisons use -cmatch and -clike

Now we can solve the rest of your problem using a pair of loops and arrays.

You'll need to create a $names array by typing out all 14 names.

$names = @("NAME1", "NAME2", "NAME3", ...)

Luckily, your example names have a nice pattern, so we can use that

$ctr = 0
$names = @("NAME") * 14 | ForEach-Object {$_     $ctr}

We'll need another array containing the months in our Regular Expressions

$quarts = @("JAN|FEB|MAR", "APR|MAY|JUN", "JUL|AUG|SEP", "OCT|NOV|DEC")

And now we shall loop

$ctr = 0
$names = @("NAME") * 14 | Foreach-Object {$_     $ctr}

$quarts = @("JAN|FEB|MAR", "APR|MAY|JUN", "JUL|AUG|SEP", "OCT|NOV|DEC")

$container = "C:\Powershell\Attempt"
$files = Get-ChildItem -Path $container -Filter *.txt
foreach ($name in $names)
{
    $quart_num = 1
    foreach ($quart in $quarts)
    {
        $files |
            Where-Object {$_.name -match "${name}\D.*(${quart})"} |
            Get-Content |
            Out-File "${container}\test\${name}_Q${quart_num}.txt" -Encoding ascii
        $quart_num  = 1
    }
}

# Remove any empty files
Get-ChildItem -Path "${container}\test" | Where-Object {$_.Length -eq 0} | Remove-Item

Note that the script will error out if the path ${container}\test doesn't exist.
Note that I've slightly changed the Regular expression in the script used - the regular expressions look like NAME1\D.*(JAN|FEB|MAR) instead of NAME1.*(JAN|FEB|MAR). This is so that a file named NAME14_JAN.txt doesn't match the regular expression corresponding to NAME1 as well as NAME14

CodePudding user response：

If there are only a few files, what you have looks pretty good to me. But if there are a large amount of files or the files are large, you have a couple places where you can gain some performance.

First is by using the -Filter parameter, like this:

Get-childitem -Path 'C:\Powershell\Attempt\*' -Filter '*NAME1*.txt'

[NOTE: The -Filter parameter usually only works well on one filter, so you'll still want to use the where-object {$_.name -like... for the different months]

The second place you can gain some performance is by setting the 'C:\Powershell\Attempt\*' -Filter '*NAME1*.txt' command equal to a variable. Setting the results to a variable allows you to make the search once and then reuse the results:

$name1 = Get-childitem -Path 'C:\Powershell\Attempt\*' -Filter '*NAME1*.txt'

You can put this into a loop if desired but in my opinion it isn't worth the effort. You can however get rid of the blank text files with some sort of existence check. Here is how the entire thing could look:

$name1 = Get-childitem -Path 'C:\Powershell\Attempt\*' -Filter '*NAME1*.txt'

    if (1 -eq ($name1 | where-object {$_.name -like "*JAN*" -or $_.name -like "*FEB*" -or $_.name -like "*MAR*"}).Count){
        get-content | Out-File C:\Powershell\Attempt\test\NAME1_Q1_.txt -Encoding ascii}
    if (1 -eq ($name1 | where-object {$_.name -like "*APR*" -or $_.name -like "*MAY*" -or $_.name -like "*JUN*"}).Count){
        get-content | Out-File C:\Powershell\Attempt\test\NAME1_Q2_.txt -Encoding ascii}
    if (1 -eq ($name1 | where-object {$_.name -like "*JUL*" -or $_.name -like "*AUG*" -or $_.name -like "*SEP*"}).Count){
        get-content | Out-File C:\Powershell\Attempt\test\NAME1_Q3_.txt -Encoding ascii}
    if (1 -eq ($name1 | where-object {$_.name -like "*OCT*" -or $_.name -like "*NOV*" -or $_.name -like "*DEC*"}).Count){
        get-content | Out-File C:\Powershell\Attempt\test\NAME1_Q4_.txt -Encoding ascii}