What is the fastest way to rename csv header names using PowerShell?-CodePudding

I am trying to automate the process of rename the csv header names. I used the following script, but the script is taking very long time to complete the rename process especially when the file size is very large. How can we optimize the script speed?

$Data=Get-Content -Path "C:/csv/lead.csv"
$Data[0] = "sr_no,dob,email,father_name,mobile,other_mobile_no.,pincode"
$Data | Out-File -FilePath "C:/csv/lead.csv" -Encoding UTF8

CodePudding user response：

You could change the first line to:

$Data= [IO.File]::ReadAllLines("C:/csv/lead.csv")

and the script should do the same thing, but run faster. You may need to specify the encoding, e.g.

$Data = [IO.File]::ReadAllLines("C:/csv/lead.csv", [Text.Encoding]::UTF8)

CodePudding user response：

For best performance, replace all PowerShell pipeline commands by direct calls to .NET API.

$Data = [IO.File]::ReadAllLines("C:/csv/lead.csv")
# Alternatively use the overload that has an encoding parameter
#$Data = [IO.File]::ReadAllLines("C:/csv/lead.csv", [Text.Encoding]::UTF8)

$Data[0] = "sr_no,dob,email,father_name,mobile,other_mobile_no.,pincode"

[IO.File]::WriteAllLines("C:/csv/lead.csv", $Data, [Text.Encoding]::UTF8)

The performance difference is astonishing. With an 80 MB CSV file, your code took 311 seconds, while the above code took just 0.8 seconds!

This is mainly due to the overhead of the PowerShell pipeline processing and to a lesser degree due to PowerShell being a scripting language.