I did a recent migration of my company's legacy app from packages.config to PackageReferences. We have 5 projects: the main ASP.NET web app, a SQL connector model project, an xUnit test project, a FluentAssertions test project, and a SpecFlow project (and no I did not set this all up). My current goal was to move all of the packages.config to properly use NuGet in PackageReference in the csproj files for an eventual move from .NET Framework 4.6.1 to .NET Core. Unfortunately, we are not ready for such a move yet.

I have done the migration for all the projects. With some fiddling, all of them build and most run correctly. Our web project builds and runs (we still need proper smoke testing but it looks good so far). Our FluentAssertions and XUnit projects also build and run all of their tests flawlessly. We do have some warnings, but there is less of them then there was before this migration.

What is not working is the SpecFlow tests. Specifically, the SpecFlow tests are not being populated in the Test Explorer automatically in Visual Studio 2019. They were before this migration. We need these tests to run (for now) in our automated build process. We are fixing our technical debt in stages.

I have investigated online for the past couple of days and can make the following claims about our SpecFlow project:

1. We have NUnit3 added in the Extensions and it appears to be "functioning". Same with SpecFlow for Visual Studio 2019 extension.
2. We have all of the NUnit3, MSTest, and SpecFlow NuGet packages from before the migration (in the same versions) as NuGet references post-migration.
3. If I run the tests from the project's context menu, it says there are 0 tests run.

Executing all tests in project: MyCompany.Specs
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========

4. I have turned on Tracing for SpecFlow in the Tools->Options->SpecFlow menu.
5. I reinstalled NUnit3 extension numerous times.
6. I have removed the <Target Name="EnsureNuGetPackageBuildImports" > tag in the csproj file to get this to build since I did have the NuGet packages. Apparently, after the migration, this tag and child elements is unnecessary because the packages are centralized.
7. I do have the import tags still in my csproj file:

    <Import Project="..\packages\MSTest.TestAdapter.2.0.0\build\net45\MSTest.TestAdapter.props" Condition="Exists('..\packages\MSTest.TestAdapter.2.0.0\build\net45\MSTest.TestAdapter.props')" />
    <Import Project="..\packages\NUnit3TestAdapter.3.10.0\build\net35\NUnit3TestAdapter.props" Condition="Exists('..\packages\NUnit3TestAdapter.3.10.0\build\net35\NUnit3TestAdapter.props')" />

8. Clean and rebuild project or solution has no effect. The other project build and run their tests correctly.
9. App.config in the Specs project has the following tag in the ConfigSection: <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" /> ...and has the following section down below in the config file:

<specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
    <unitTestProvider name="NUnit" />
</specFlow>

Any idea why SpecFlow tests would no longer show up in Test Explorer? Anything else I should check to diagnose this problem? Is there a known issue with SpecFlow with migrating from packages.config to PackageReference?

CodePudding user response：

1. If you still have the Visual Studio Extension (VSIX) for the NUnit Visual Studio Test Adapter (and it appears you do from your comment "We have NUnit3 added in the Extensions"), you need to disable or remove it. The VSIX adapter will be used over the Nuget packaged adapter and causes conflicts with the Nuget packaged adapter. The VSIX extension also shouldn't be used in VS 2017 or higher. You can check in Main Menu -> Tools-> Extensions and Updates and look for Nunit3TestAdapter.

   Using VSIX extensions was the old way to use test adapters and runners. It was brittle because you had one version installed in VS that needed to support many different projects and the VS extensions framework is pretty convoluted with awkward hooks into the build life cycle. The new way is to package test adapters and runners as NuGet packages. There is nothing to install and Visual Studio finds the version-specific test adapter and runner libraries for your project besides your test assemblies. You need to use one or the other, and Nuget packages are preferred now, or Visual Studio will fight against you and you'll have issues similar to what you are seeing.

2. The Visual Studio Test Explorer cache sometimes gets out of whack. Try closing all copies of Visual Studio and cleaning its temp files and folders at C:\Users{$username}\AppData\Local\Temp\VisualStudioTestExplorerExtensions\

3. You might be running into an issue where the test process is running on the is selecting the wrong architecture of the adapter or other library during test discovery orchestration. Try changing the default processor architecture for tests (Main Menu Test -> Test Settings). Try changing x86 to x64 or vice versa.

CodePudding user response：

Thank you @Dude0001 for your answer. It was partially correct, but it wasn't the whole tale.

When you migrate a SpecFlow project from packages.config to PackageReference, you are trading the local NuGet packages and settings for global settings in NuGet because of this, we no longer need VSIX extensions for SpecFlow (as Dude0001 stated above).

Along with that, I had to remove all of the configuration in App.config and the csproj files for the import statements and the other configuration for specflow specifically. You don't need to specify what test runner you are using with SpecFlow (NUnit in my case).

As well, I had to update NUnit and SpecFlow and their supporting NuGet packages to the latest versions for .NET Framework 4.6.1.

All of this combined to bring me to success to find the tests and run them correctly as they did before the migration.