Powershell: How to remove all files in Directory except recent ones-CodePudding

Have folder which has backups of SQL databases with backup date in the name. e.g. C:\Backup folder.
Example of backup files:
archive_1_01022022.bak
archive_1_02022022.bak
archive_1_03022022.bak
archive_2_01022022.bak
archive_2_02022022.bak
archive_2_03022022.bak
archive_3_01022022.bak
archive_3_02022022.bak
archive_3_03022022.bak

I need powershell script which removes all files from this directory but keeps recent ones (e.g. for last 5 days), but at the same time I need to keep at least 3 copies of each database (in case there are no backups done for more than last 5 days).

Below script removes all files and keeps recent ones for last 5 days:

$Folder = "C:\Backup"
$CurrentDate = Get-Date 
$DateDel = $CurrentDate.AddDays(-5) 
Get-ChildItem $Folder | Where-Object { $_.LastWriteTime -lt $DateDel } | Remove-Item

Above is wokring fine, but if there are no recent backups for last 10 days and if I run above code then it will remove all files in C:\Backup. For such cases I need to keep at least 3 backup files of each databases.

If I use below code (for example I have 9 different databases), then it do job:

$Folder = "C:\Backup"
Get-ChildItem $Folder | ? { -not $_.PSIsContainer } |
Sort-Object -Property LastWriteTime -Descending |  
Select-Object -Skip 27 | 
Remove-Item -Force

But implementation is weird. For example if I have backups of 9 databases, then I need to provide "Select-Object -Skip" with value 27 (9 databases x skip 3 files of each database). In case I have more databases or less, then each time I need to adjust this number. How can I make "Select-Object -Skip 3" static value?

CodePudding user response：

In that case, you need to test how many files with a newer or equal date compared to the reference date there are in the folder. If less than 3, sort them by the LastWriteTime property and keep the top 3. If you have enough newer files left, you can delete the old ones:

$Folder  =  "C:\Backup"
$DateDel = (Get-Date).AddDays(-5).Date  # set to midnight

# get a list of all backup files
$allFiles = Get-ChildItem -Path $Folder -Filter 'archive*.bak' -File

# test how many of these are newer than 5 days ago
$latestFiles = @($allFiles | Where-Object { $_.LastWriteTime -ge $DateDel })
if ($latestFiles.Count -lt 3) {
    # if less than three keep the latest 3 files and remove the rest
    $allFiles | Sort-Object LastWriteTime -Descending | Select-Object -Skip 3 | Remove-Item -WhatIf
}
else {
    # there are plenty of newer files, so we can remove the older ones
    $allFiles | Where-Object { $_.LastWriteTime -lt $DateDel } | Remove-Item -WhatIf
}

^{I have added the -WhatIf safety switch to both Remove-Item cmdlets, so you can first see what would happen before actualy destroying files. Once you are satisfied with what the console shows, remove those -WhatIf switches and run again}

If you have 9 databases and the number in the filename after archive_ makes the distinction between those database backup files, just put the above inside a loop and adjust the -Filter:

$Folder  = "C:\Backup"
$DateDel = (Get-Date).AddDays(-5).Date  # set to midnight
# loop through the 9 database files
for ($i = 1; $i -le 9; $i  ) {
    # get a list of all backup files per database
    $allFiles = Get-ChildItem -Path $Folder -Filter "archive_$($i)_*.bak" -File

    # test how many of these are newer than 5 days ago
    $latestFiles = @($allFiles | Where-Object { $_.LastWriteTime -ge $DateDel })
    if ($latestFiles.Count -lt 3) {
        # if less than three keep the latest 3 files and remove the rest
        $allFiles | Sort-Object LastWriteTime -Descending | Select-Object -Skip 3 | Remove-Item -WhatIf
    }
    else {
        # there are plenty of newer files, so we can remove the older ones
        $allFiles | Where-Object { $_.LastWriteTime -lt $DateDel } | Remove-Item -WhatIf
    }
}

Ok, so now we know the example names you gave do not bare resemblance with the real names, the code could be as simple as this:

$dbNames = 'archive', 'master', 'documents', 'rb'  # the names used in the backup files each database creates
$Folder  = "C:\Backup"
$DateDel = (Get-Date).AddDays(-5).Date  # set to midnight
# loop through the database files
foreach ($name in $dbNames) {
    # get a list of all backup files per database
    $allFiles = Get-ChildItem -Path $Folder -Filter "$($name)_*.bak" -File

    # test how many of these are newer than 5 days ago
    $latestFiles = @($allFiles | Where-Object { $_.LastWriteTime -ge $DateDel })
    if ($latestFiles.Count -lt 3) {
        # if less than three keep the latest 3 files and remove the rest
        $allFiles | Sort-Object LastWriteTime -Descending | Select-Object -Skip 3 | Remove-Item -WhatIf
    }
    else {
        # there are plenty of newer files, so we can remove the older ones
        $allFiles | Where-Object { $_.LastWriteTime -lt $DateDel } | Remove-Item -WhatIf
    }
}

CodePudding user response：

Basing on the assumption that your backups have a name convention of : DBNAME_ddMMyyyy.bak where the date correspond to the backup date, I would do something like below.


$Params = @{
    MinBackupThresold = 1
    MinBackupDays = 5
    SimulateDeletion = $False # Set to true to perform a Remove-Item -WhatIf deletion}


$Folder = "C:\temp\test"
$CurrentDate = Get-Date 
$DateDel = $CurrentDate.AddDays($Params.MinBackupDays).Date # set to midnight


$Archives = Foreach ($File in  Get-ChildItem $Folder ) {
    # -13 come from assuming naming convention DBName_8CharBackupDate.ext (eg: Db1_01012022.bak)
    $DbNameEndIndex = $File.Name.Length - 13
    #  1 since our naming convention have an underscore between db name and date.
    $RawDateStr = $File.Name.Substring($DbNameEndIndex   1 , 8)
    [PSCustomObject]@{
        Path = $FIle.FullName
        LastWriteTime = $File.LastWriteTime
       DBName        = $File.Name.Substring(0, $DbNameEndIndex)
       BackupDate    = [datetime]::ParseExact( $RawDateStr, 'ddMMyyyy', $null)
    }
}

#Here we group archives by their "dbname" so we can make sure to keep a min.  backups for each.
$GroupedArchives = $Archives | Group DBName
Foreach ($Db in $GroupedArchives) {

    if ($Db.Count -gt $Params.MinBackupThresold) {
        $Db.Group | Sort BackupDate | Select-Object -Skip $Params.MinBackupThresold | Where-Object { $_.BackupDate -lt $DateDel } | % { Remove-Item -Path $_.Path -Force -WhatIf:$Params.SimulateDeletion }
    } else {

        # You could include additional checks to verify last backup, alert you if there should be more in there, etc...
    }

}

Note: Using the date extracted from the filename will be more accurate than the lastwritetime, which could be updated for other reasons (Since we have it, might as well use it.)

Note 2 : Added WhatIf in the $params so you can easily switch between actual removal and simulation (Theo's answer gave me the idea of providing that switch) and his .Date to make sure the date was set to midnight instead of current time of day.