Resetting color escape sequences in PowerShell-CodePudding

I have a PowerShell script that calls out to sqlcmd. After it returns, it seems that it has disabled all the color output in PowerShell (v7.2.0) and I'm getting the ANSI escape sequences output in the window (e.g. [33;1m) I've tried calling [Console]::ResetColor() but that doesn't seem to do anything.

Any ideas how to tell Powershell to start interpreting the color escape sequences instead of printing the extra garbage in the console?

Example the reproduces the problem:

Write-Warning "hello world" 
$process = Start-Process sqlcmd -ArgumentList @('-Q',"`"$($Command)`"") -PassThru -NoNewWindow -Wait
Write-Warning "hello again"

CodePudding user response：

Thanks to @Jeroen for the breadcrumbs.

Based on that I have the following solution:

foo.ps1:

.".\Reset-ConsoleColors.ps1"
Write-Warning "hello world" 
$process = Start-Process sqlcmd -ArgumentList @('-Q',"`"$($Command)`"") -PassThru -NoNewWindow -Wait
Reset-ConsoleColors
Write-Warning "hello again"

Reset-ConsoleColors.ps1:

function Reset-ConsoleColors {
    $MethodDefinitions = @'
[DllImport("kernel32.dll", SetLastError = true)]
public static extern IntPtr GetStdHandle(int nStdHandle);
[DllImport("kernel32.dll", SetLastError = true)]
public static extern bool GetConsoleMode(IntPtr hConsoleHandle, out uint lpMode);
[DllImport("kernel32.dll", SetLastError = true)]
public static extern bool SetConsoleMode(IntPtr hConsoleHandle, uint lpMode);
'@
    $Kernel32 = Add-Type -MemberDefinition $MethodDefinitions -Name 'Kernel32' -Namespace 'Win32' -PassThru
    $hConsoleHandle = $Kernel32::GetStdHandle(-11) # STD_OUTPUT_HANDLE 
    $mode = 7
    $Kernel32::SetConsoleMode($hConsoleHandle, $mode) | out-null
}

Still no idea why sqlcmd is changing the console mode or how to tell it not to, but now the console output doesn't go crazy after I call it.

CodePudding user response：

While the symptom is an interesting one, your feedback tells us that the problem can easily be avoided, and doing so isn't just a workaround, it's actually the superior approach to running console applications synchronously, in the current window:

Instead of Start-Process:

$process = Start-Process sqlcmd -ArgumentList @('-Q',"`"$($Command)`"") -PassThru -NoNewWindow -Wait

use direct invocation:

$output = sqlcmd -Q $Command

Per your feedback, this made the problem go away (interpretation of Virtual Terminal / ANSI escape sequences continued to work).

In addition to running synchronously in the current window, direct invocation:

connects the external program's standard output streams to PowerShell's streams, allowing you capture stdout output, as shown above, and to redirect stderr output (with 2>)
reflects the external program's (process') exit code in the automatic $LastExitCode variable

Only in unusual cases is Start-Process the right tool for invoking console applications, such as wanting to run in a new window (without -NoNewWindow) - see this answer. GitHub docs issue #6239 provides guidance on when use of Start-Process is and isn't appropriate.