I am writing a script that searches for a specific pattern in multiple log files with Powell Shell. There is a serious memory leak when executing the script, so I am wondering how to search the files one by one. Help.

$LOG_FILE = 'C:\test\test\test_*.log
$FIND_LINE = Get-Content $LOG_FILE | findstr 'test'

CodePudding user response：

Get-Content is slow and memory-hungry, so it's best to let Select-String - PowerShell's more fully-featured analog to findstr.exe - search the files of interest directly:

$linesFound = (Select-String -Path C:\test\test\test_*.log -Pattern 'test').Line

Note:

• By default, the -Pattern argument is interpreted as a regex; add the -SimpleMatch switch to perform a verbatim substring search.

• Select-String, like PowerShell in general, is case-insensitive by default;
  add -CaseSensitive if needed.

• Select-String wraps the lines it finds in [Microsoft.PowerShell.Commands.MatchInfo] instances that contain metadata about each match; accessing the .Line property returns the matching line.

  • In PowerShell (Core) 7 you can use the -Raw switch to directly request the matching lines only.

CodePudding user response：

Get the log files into an array and then loop through them. Don't use findstr as that is not native powershell its an old cmd executable. Use Select-String as others have said.

[array]$LOG_FILE = Get-ChildItem C:\test\test\test_*.log
[string[]]$FIND_LINE = ForEach ($L in $LOG_FILE) { Get-Content $L | Select-String 'test' }

Then you could export the results to a text file like so:

$FIND_LINE | Out-File "C:\test\FoundLines.txt"

Since you mentioned memory leaks and such. Powershell can eat memory it never returns within a given session but if you save this into a script and execute it as a script then it should clear up memory after the script ends. However there are a few tricks to clean up memory within loops like this dotnet command for garbage collection:

[system.gc]::Collect()

It will add a tiny bit of processing time to each iteration of the loop though so I would test how long it takes with and without the garbage collection to make sure it's worth it. I suggest also reading about some other methods and caveats with that mentioned here: http://www.jhouseconsulting.com/2017/09/25/addressing-the-powershell-garbage-collection-bug-1825

And also use PowerShell 7 (if you can install it) as it would likely be faster and more efficient.