PowerShell, Cannot remove BOM from file-CodePudding

I have been trying dozens of different options for an hour, strip the BOM, set file to ascii / oem / utf8, followed the ideas here, but the BOM remains in all cases. I'm a bit stumped.

Firstly, I always Remove-Item with -Force to make sure that there is no remnant of the file before I start, then I pipe some simple text into a newly created file:

$out = @'
####################
#
# Shortcuts
#
####################
'@
$OutputEncoding = [Text.Utf8Encoding]::new($false)
$out | Out-File -FilePath $ShortcutNotes -Force -Encoding utf8
# I have tried utf8 / oem / ascii

This part of text always looks fine in the final file, no strangeness.

But then I pipe some fairly innocuous output from some operations into the file and everything goes wrong:

# Some operations before here, but this is the important line:
$AppName = (($AppShortcut -split "\\")[-1] -split "\.")[0]
"App added: $AppName" | Out-File -FilePath $ShortcutNotes -Append

End result, I get UTF8 with BOM when I open the file in Notepad and it looks like this (header part is always fine, then the rest has the NULLs):

####################
#
# Shortcuts
#
####################

A[NULL]p[NULL]p[NULL] [NULL]a[NULL]d[NULL]d[NULL]e[NULL]d[NULL]:[NULL] [NULL]G[NULL]o[NULL]o[NULL]g[NULL]l[NULL]e[NULL] [NULL]C[NULL]h[NULL]r[NULL]o[NULL]m[NULL]e

I've even tried to strip BOM afterwards:

(Get-Cotent $ShortcutNotes) -replace "\xEF\xBB\xBF", "" | Set-Content $ShortcutNotes

All of the above fails, and the file is always UTF8 with BOM and always garbled as above.

I've run out of things to try, can anyone see what I am doing wrong that my output is always garbled in this way?

Update The operations that I'm doing after the header are very simple, but maybe they will suggest what is going wrong. This is a snippet from the function that is called and that writes to the file. It seems to be this that is causing the issue, but I don't know how/why:

function Setup-AppAndShortcut ($AppFolder, $ZipFile, $AppExe, $AppShortcut) {
    if (Test-Path $AppExe) {
        $ShortcutFile = $AppShortcut
        $Shortcut = $WScriptShell.CreateShortcut($ShortcutFile)
        $Shortcut.TargetPath = $AppExe
        $Shortcut.Save()
        [string]$AppName = (($AppShortcut -split "\\")[-1] -split "\.")[0]
        "App added: $AppName" | Out-File -FilePath $ShortcutNotes -Append   # These are the lines that always end up garbled with NULLs in my final output.
    }

CodePudding user response：

Use add-content instead of out-file -append, which can mix different encodings in the same file. Bug report:

out-file -append (or >>) can mix two encodings in the same file · Issue #9423 · PowerShell · GitHub