PowerShell Core: resolving assembly conflicts-CodePudding

I have found an interesting case which I can't fully understand.

My runtime - PowerShell 7.2.4

I have two PowerShell Core modules: Net6PowerShellModule and NetstandardPowerShellModule.

Net6PowerShellModule.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
  </PropertyGroup>

  <ItemGroup>
    <None Remove="Net6PowerShellModule.psd1" />
  </ItemGroup>

  <ItemGroup>
    <Content Include="Net6PowerShellModule.psd1">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </Content>
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="6.0.0" />
    <PackageReference Include="System.Management.Automation" Version="7.2.4" />
  </ItemGroup>
</Project>

Test-Net6Cmdlet.cs

using System.Management.Automation;

namespace Net6PowerShellModule
{
    [Cmdlet("Test", "Net6Cmdlet")]
    public class TestNet6Cmdlet : PSCmdlet
    {
        protected override void ProcessRecord()
        {
            var type = typeof(Microsoft.Extensions.DependencyInjection.IServiceCollection);

            WriteObject("Net6 Cmdlet Run.");
        }
    }
}

Net6PowerShellModule.psd1

@{
    RootModule = 'Net6PowerShellModule.dll'
    ModuleVersion = '0.0.1'
    GUID = '8c1bd929-32bd-44c3-af6b-d9dd261e34f3'
    CmdletsToExport = 'Test-Net6Cmdlet'
}

NetstandardPowerShellModule.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
  </PropertyGroup>

  <ItemGroup>
    <None Remove="NetstandardPowerShellModule.psd1" />
  </ItemGroup>

  <ItemGroup>
    <Content Include="NetstandardPowerShellModule.psd1">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </Content>
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="2.2.0" />
    <PackageReference Include="System.Management.Automation" Version="6.1.2" />
  </ItemGroup>
</Project>

Test-NetstandardCmdlet.cs

using System.Management.Automation;

namespace NetstandardPowerShellModule
{
    [Cmdlet("Test", "NetstandardCmdlet")]
    public class TestNetstandardCmdlet : PSCmdlet
    {
        protected override void ProcessRecord()
        {
            var type = typeof(Microsoft.Extensions.DependencyInjection.IServiceCollection);

            WriteObject("Netstandard 2.0 Cmdlet Run.");
        }
    }
}

NetstandardPowerShellModule.psd1

@{
    RootModule = 'NetstandardPowerShellModule.dll'
    ModuleVersion = '0.0.1'
    GUID = 'eef615d0-aecf-4e89-8f4c-53174f8c99d6'
    CmdletsToExport = 'Test-NetstandardCmdlet'
}

Next, I have two possible use cases for those modules:

Import Net6PowerShellModule
Import NetstandardPowerShellModule
Run Test-Net6Cmdlet
Run Test-NetstandardCmdlet

Result - everything runs correctly. The only loaded assembly is Microsoft.Extensions.DependencyInjection.Abstractions version 6.0.0.

Import Net6PowerShellModule
Import NetstandardPowerShellModule
Run Test-NetstandardCmdlet
Run Test-Net6Cmdlet.

Result - error: Test-Net6Cmdlet: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. Could not find or load a specific file. (0x80131621)

Based on this question, this is not an expected behavior:

Subsequent attempts to load a different version of the assembly: cause a statement-terminating error in Powershell (Core) 7 Assembly with same name is already loaded.

Why does the first scenario work then?

If I add the RequiredAssemblies = @('Microsoft.Extensions.DependencyInjection.Abstractions.dll') line in both module manifests, I got an expected behavior: regardless of which module is loaded first, I still got an error: Assembly with same name is already loaded.

Update 1

Articles that I've already looked at:

https://docs.microsoft.com/en-us/powershell/scripting/dev-cross-plat/resolving-dependency-conflicts
PowerShell: why am I able to load multiple versions of the same .NET assembly in one session and also "bypass Dependency hell"?
https://devblogs.microsoft.com/powershell/resolving-powershell-module-assembly-dependency-conflicts/ - this article explains a lot about Assembly Load Contexts, but I still cannot understand the difference between having RequiredAssemblies defined in .psd1 files or using the dependencies in code.

CodePudding user response：

tl;dr

When authoring a module, it is best to use a module manifest (.psd1 file) with a RequiredAssemblies key to declare dependent (helper) assemblies, because it makes loading of such assemblies predictable, because it happens at the time of module import.
- In Windows PowerShell, attempts to load a different version of an already-loaded assembly are quietly ignored and the newly imported module is made to use the already-loaded assembly too, which may or may not work.
- In PowerShell Core (v6 ), attempts to load a different version of an already-loaded assembly are categorically prevented.
If you use implicit loading of dependent assemblies, based on .NET's mechanisms, the timing of when these dependencies are loaded isn't guaranteed, and, in PowerShell Core, you'll bypass the check that prevents attempts to load a different version of an already-loaded assembly - which may or may not work and is only allowed if the later load attempt requests a lower version than the one already loaded.
Workarounds are needed to reliably load different versions of a given assembly side by side.

Background information

One of your links, Resolving PowerShell module assembly dependency conflicts, contains all relevant information, but it's a lengthy read with complex, in-depth information, and it doesn't clarify the way in which PowerShell (Core) (v6 ) overrides the behavior of the underlying .NET (Core) / .NET 5 framework.

Terminology: For simplicity, I'll use the following terms:

Windows PowerShell is the legacy, ships-with-Windows Windows-only edition of PowerShell whose latest and final version is 5.1.x.
- It is built on top of .NET Framework, the legacy, Windows-only edition of .NET, whose latest and last version is v4.8.x
PowerShell Core (officially now just PowerShell) is the modern, cross-platform, install-on-demand PowerShell edition, whose first (but now obsolete) version was v6 and whose current version as of this writing is v7.2.4.
- It is built on top of .NET Core (officially since v5 just .NET), the modern, cross-platform, install-on-demand edition of .NET. which builds on .NET Framework and PowerShell Core.

The underlying problem:

At the .NET framework / CLR (Common Language Runtime) level, the fundamental problem is the inability to load different versions of a given assembly side by side into the same application domain (.NET Framework) / ALC (assembly-load context, .NET Core): the version that is loaded first into a session quietly wins, with .NET Core imposing the additional restriction that only attempts to load lower versions succeed (see below).
PowerShell uses only a single application domain / ALC, including for modules, necessitating workarounds to enable side-by-side loading of different versions of a (dependent) assembly. The linked article details various workarounds; a robust, generic, in-process workaround requires PowerShell Core and is nontrivial; limited workarounds, some out-of-process, are available for Windows PowerShell and cross-PowerShell-edition code.

Behavior without a workaround:

Windows PowerShell and .NET Framework:
- .NET Framework quietly ignores subsequent attempts to load a different version of an already-loaded assembly, and makes the caller use the already-loaded version, irrespective of whether the requested version is higher or lower than the already loaded one.
  - Curiously, when the caller uses reflection with typeof to inspect the assembly and its types it just loaded (whether implicitly or explicitly), the requested assembly version is reported - even though it is the already loaded assembly and its types that are actually used.
- Windows PowerShell does not overlay this behavior with custom functionality, unlike .NET Core, although whether or not use a module manifest (.psd1 file) with a RequiredAssemblies entry can make a difference with respect to when dependent assemblies are loaded (see below)
- Upshot:
  - Attempts to load a different version of an already-loaded assembly never fail and quietly make the caller use the already-loaded version.
  - This may or may not work, and you'll only notice at the time the types from the assembly in question are actually used (construction of an instance, property access, method calls, ...).
    - Notably, it is likely to fail if a later caller attempts to load a higher version of an assembly, as it may rely on features that the already-loaded lower version doesn't have, but even the inverse scenario isn't guaranteed to succeed.
PowerShell Core and .NET Core:
- .NET Core:
  - Quietly ignores subsequent attempts to load a lower version of an already-loaded assembly, and makes the caller use the already-loaded version
  - Throws an exception for attempts to load a higher version: Could not find or load a specific file. (0x80131621)
  - The rationale appears to be: an already-loaded higher version is at least likely to be backward-compatible with the requested lower version - although, in the absence of semantic versioning for .NET assemblies - this is not guaranteed.
  - When no exception occurs, and the caller uses reflection with typeof to inspect the assembly and its types it just loaded (whether implicitly or explicitly), it does see the actual, already-loaded assembly version - unlike in .NET Framework.
- PowerShell Core does overlay this behavior with custom functionality, unlike Windows PowerShell:
  - It defines a custom dependency handler for the plugin-oriented [System.Reflection.Assembly]::LoadFrom() .NET Core API, which it uses behind the scenes in the following contexts:
    - When loading a module with a module manifest (.psd1) that declares dependent assemblies via its RequiredAssemblies entry.
    - Calls to Add-Type with -Path / -LiteralPath.
    - in using assembly statements.
  - This custom dependency resolution handler categorically prevents attempts to load a different version of an already-loaded version, irrespective of which version number is higher.
    - A statement-terminating Assembly with same name is already loaded error occurs (in the case of using assembly, loading a script breaks in the parsing stage with Cannot load assembly '...').
- Upshot:
  - In PowerShell Core, attempts to load a different version of an already-loaded assembly always fail if you let PowerShell make the load attempt.
    - Only if you use .NET Core's implicit dependency loading is there a chance that the load attempt succeeds, namely if an attempt is made to load a lower version later. Implicit dependency loading happens automatically, via the [System.Reflection.Assembly]::Load() .NET API, which is based on supplying a dependent assembly's (location-independent) full name and default probing paths for looking for the file hosting it, which notably includes the directory of the calling assembly itself.
    - This may or may not work, given that versioning of .NET assemblies doesn't use semantic versioning, so a higher version of an assembly isn't guaranteed to be backward-compatible.
    - Also, the timing of implicit loading of dependencies isn't predictable, and usually doesn't happen until the types from the assembly in question are actually used (construction of an instance, property access, method calls, ...). Thus, if you rely on your modules to use implicit dependency loading, there is no guaranteed relationship between when a module is imported vs. when actual dependency loading is attempted, so it may fail if a dependency with a lower version happens to be loaded first in a given session (but, as stated, even if the higher version is loaded first, you may see a later error when the types are actually used).