Powershell pull last 5 minutes of data from CSV-CodePudding

I have a log file that generates new data every five minutes, I am attempting to pull the last five minutes of the log file and pull specific data from the last five minutes. Currently I have code to convert it from .log to .csv with headers of "Date, Time, Error1, Error2, Error3". However, every attempt I have tried thus far is not pulling the data correctly.

The Date and Time of the CSV are formatted as: "dd/MM/yyyy","hh:mm:ss.ms"

Powershell does not give any visible errors, but the errorCOLLECTION.csv does not generate

The Current code I have:

Copy-Item -Path "C:\ProgramData\Blah\Blah\Blah Blah\error.log" -Destination "C:\Windows\Blah\Blah\Logs\Temp\Blah Blah\" -PassThru
    
    Import-Csv "C:\Windows\Blah\Blah\Logs\Temp\Blah Blah\error.log" -delimiter "," -Header Date, Time, Error1, Error2, Error3 |
    Export-Csv -NoTypeInformation "C:\Windows\Blah\Blah\Logs\Temp\Blah Blah\error.csv"
    
    
    $referenceTime = '{0:dd/MM/yyyy,HH:mm:ss.ms}' -f (Get-Date '2019/02/25,19:09:00.590').AddMinutes(-5)
    $regexSearch   = '\bSdata:\s*\[(\d{2})]'
    
    switch -Regex -File "C:\Windows\Blah\Blah\Logs\Temp\Blah Blah\error.csv" {
        $regexSearch { 
            if (($_ -split ',')[0] -gt $referenceTime) { 
                set-content "C:\Windows\Blah\Blah\Logs\Temp\Blah Blah\errorCOLLECTION.csv"
            }
        }
    }

In response to Theo an example of the Log file:

29/11/2022,10:48:48.693,PINSP,DC,<PID>6324</PID><TID>2996</TID><F>INFO_GET_KEY_DETAIL</F><X>lpszKeyName [CommsKey]</X><SF>key_lib.cpp</SF><SL>1177</SL>
29/11/2022,10:48:48.693,MDMSP,DC,<PID>6200</PID><TID>5772</TID><G><X>W</X><HS>65535</HS><RI>0</RI></G><F>SXIO::ReceiveIOMessage</F><AR><AN>RetrieveMessage</AN><RV><I4>0</I4></RV><P><N>szMessage</N><S>messageCategory: 0x3 messageType: 0x554cc006 messageID: 0xd6d7928</S></P><P><N>response</N><S>hservice: 43 ucClass: 3 usTLen: 4 TData: [33 01 00 12] ucSLen: 1 SData: [00] ucMLen: 0 ucRSlen: 0 ucRClen: 0</S></P></AR><SF>SXIO.cpp</SF><SL>833</SL>
29/11/2022,10:48:48.693,PINSP,DC,<PID>6324</PID><TID>2996</TID><F>INFO_GET_KEY_DETAIL</F><X>Return Value [0]</X><X>lptKeyDetail [caKeyName [CommsKey] usKeyId [2] usKeyspaceId [4] wTypeOfAccess [0x2] bIsIV [0] bMasterKey [0] bLoadedFlag [1] bIsDouble [0]]</X><SF>key_lib.cpp</SF><SL>1214</SL>
29/11/2022,10:48:48.693,PINSP,DC,<PID>6324</PID><TID>2996</TID><F>INFO_GET_KEY_DETAIL</F><X>lpszKeyName [MACKey]</X><SF>key_lib.cpp</SF><SL>1177</SL>
29/11/2022,10:48:48.693,PINSP,DC,<PID>6324</PID><TID>2996</TID><F>INFO_GET_KEY_DETAIL</F><X>Return Value [0]</X><X>lptKeyDetail [caKeyName [MACKey] usKeyId [3] usKeyspaceId [3] wTypeOfAccess [0x4] bIsIV [0] bMasterKey [0] bLoadedFlag [0] bIsDouble [1]]</X><SF>key_lib.cpp</SF><SL>1214</SL>
29/11/2022,10:48:48.694,PINSP,DC,<PID>6324</PID><TID>2996</TID><F>INFO_GET_KEY_DETAIL</F><X>lpszKeyName [PEKey]</X><SF>key_lib.cpp</SF><SL>1177</SL>
29/11/2022,10:48:48.694,PINSP,DC,<PID>6324</PID><TID>2996</TID><F>INFO_GET_KEY_DETAIL</F><X>Return Value [0]</X><X>lptKeyDetail [caKeyName [PEKey] usKeyId [4] usKeyspaceId [4] wTypeOfAccess [0x2] bIsIV [0] bMasterKey [0] bLoadedFlag [0] bIsDouble [1]]</X><SF>key_lib.cpp</SF><SL>1214</SL>
29/11/2022,10:48:48.694,PINSP,FW,<PID>6324</PID><TID>2996</TID><G><X>W</X><HS>44</HS><RI>2267</RI></G><F>PostWFSResult</F><F>WFSResultData</F><P><N>hWnd</N><PT>263508</PT></P><P><N>lpWFSResult->RequestID</N><U4>2267</U4></P><P><N>lpWFSResult->hService</N><U4>44</U4></P><P><N>lpWFSResult->hResult</N><H>0</H></P><P><N>lpWFSResult->u.dwCommandCode</N><U4>401</U4></P><P><N>lpStatus</N><OB><M><N>fwDevice</N><U2>0</U2></M><M><N>fwEncStat</N><U2>0</U2></M><M><N>lpszExtra</N><PT>00000000</PT></M><M><N>guidlight</N><S>0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0</S></M><M><N>fwAutoBeepMode</N><U2>2</U2></M><M><N>dwCertificateState</N><U4>4</U4></M><M><N>wDevicePosition</N><U2>3</U2></M><M><N>usPowerSaveRecoveryTime</N><U2>0</U2></M><M><N>wAntiFraudModule</N><U2>0</U2></M></OB></P><SF>FWResultImpl.cpp</SF><SL>4638</SL><E>WFS_GETINFO_COMPLETE</E>
29/11/2022,10:48:48.697,PINSP,FW,<PID>6324</PID><TID>2996</TID><G><X>W</X><HS>44</HS><RI>2268</RI></G><F>PostWFSResult</F><F>WFSResultData</F><P><N>hWnd</N><PT>7014346</PT></P><P><N>lpWFSResult->RequestID</N><U4>2268</U4></P><P><N>lpWFSResult->hService</N><U4>44</U4></P><P><N>lpWFSResult->hResult</N><H>0</H></P><P><N>lpWFSResult->u.dwCommandCode</N><U4>408</U4></P><SF>FWResultImpl.cpp</SF><SL>4638</SL><E>WFS_GETINFO_COMPLETE</E>
29/11/2022,10:48:48.702,Mgr,Mgr,<PID>6324</PID><TID>6588</TID><G><HS>44</HS></G><F>WFSGetInfo</F><P><N>*lppResult</N><OB><M><N>RequestID</N><U4>2268</U4></M><M><N>hService</N><U2>44</U2></M><M><N>hResult</N><U4>0</U4></M><M><N>Code</N><U4>408</U4></M><M><N>lpBuffer</N><PT>280E284D</PT></M></OB></P><RV><H>0</H></RV><SF>MgrApi.cpp</SF><SL>1394</SL>
29/11/2022,10:48:48.702,Mgr,Mgr,<PID>6324</PID><TID>6588</TID><G><HS>0</HS></G><F>WFSFreeResult</F><P><N>lpResult</N><OB><M><N>RequestID</N><U4>2268</U4></M><M><N>hService</N><U2>44</U2></M><M><N>hResult</N><U4>0</U4></M><M><N>Code</N><U4>408</U4></M><M><N>lpBuffer</N><PT>280E284D</PT></M></OB></P><SF>MgrApi.cpp</SF><SL>1230</SL>
29/11/2022,10:48:48.702,Mgr,Mgr,<PID>6324</PID><TID>6588</TID><G><HS>0</HS></G><F>WFSFreeResult</F><RV><H>0</H></RV><SF>MgrApi.cpp</SF><SL>1240</SL>
29/11/2022,10:48:48.703,Mgr,Mgr,<PID>6324</PID><TID>6588</TID><G><HS>0</HS></G><F>WFSFreeResult</F><P><N>lpResult</N><OB><M><N>RequestID</N><U4>2266</U4></M><M><N>hService</N><U2>49</U2></M><M><N>hResult</N><U4>0</U4></M><M><N>Code</N><U4>301</U4></M><M><N>lpBuffer</N><PT>08120D85</PT></M></OB></P><SF>MgrApi.cpp</SF><SL>1230</SL>
29/11/2022,10:48:48.703,Mgr,Mgr,<PID>6324</PID><TID>6588</TID><G><HS>0</HS></G><F>WFSFreeResult</F><RV><H>0</H></RV><SF>MgrApi.cpp</SF><SL>1240</SL>
29/11/2022,10:48:48.703,Mgr,Mgr,<PID>6324</PID><TID>6588</TID><G><HS>0</HS></G><F>WFSFreeResult</F><P><N>lpResult</N><OB><M><N>RequestID</N><U4>2267</U4></M><M><N>hService</N><U2>44</U2></M><M><N>hResult</N><U4>0</U4></M><M><N>Code</N><U4>401</U4></M><M><N>lpBuffer</N><PT>281523A5</PT></M></OB></P><SF>MgrApi.cpp</SF><SL>1230</SL>
29/11/2022,10:48:48.703,Mgr,Mgr,<PID>6324</PID><TID>6588</TID><G><HS>0</HS></G><F>WFSFreeResult</F><RV><H>0</H></RV><SF>MgrApi.cpp</SF><SL>1240</SL>

CodePudding user response：

Doing a slight modification to your code, because ($_ -split ',')[0] would be only targeting the Date and not the Time, the following works properly for me outputting the line starting with:

29/11/2022,10:48:48.693,MDMSP,DC,<PID>6200...

I'm also using DateTime.TryParse to convert these strings into a DateTime instance, I'm honestly not sure comparing these strings as strings would work correctly, at least converting them to DateTime instances we're 100% sure the comparison will be correct.

Aside from that, as was pointed out in comments by other users, Set-Content currently has the path to output but no value as argument.

& {
    $referenceTime = (Get-Date '2019/02/25,19:09:00.590').AddMinutes(-5)
    $regexSearch   = '\bSdata:\s*\[(\d{2})]'
    $parsedDate    = [ref] [datetime]::new(0)
    
    switch -Regex -File 'C:\bla\bla\error.csv' {
        $regexSearch {
            $success = [datetime]::TryParseExact(
                ('{0},{1}' -f $_.Split(',', 3)[0, 1]),
                'dd/MM/yyyy,HH:mm:ss.fff',
                [cultureinfo]::InvariantCulture,
                [System.Globalization.DateTimeStyles]::AssumeLocal,
                $parsedDate
            )
    
            if($success -and $parsedDate.Value -gt $referenceTime) {
                $_
            }
        }
    }
} | Set-Content 'C:\bla\bla\errorCOLLECTION.csv'