I am trying to set up an Azure pipeline which will run a Selenium test suite. The suite was build in VS2022 with C# and Selenium. The suite runs fine from VS, command line and also Jenkins but we are looking to build it into a release pipeline and this is a test to see how this will work.
The test pipeline appears to work until we reach task - VSTest@2. At this point the process finds the build dll but does not find the tests which are in this file, below is the current code
trigger:
- master
pool:
name: Azure Pipelines
vmImage: 'windows-latest'
demands: vstest
variables:
solution: '**/Testapp.sln'
steps:
- task: NuGetToolInstaller@1
- task: NuGetCommand@2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild@1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site"'
- task: VSTest@2
displayName: 'Regression tests - warmup'
inputs:
testSelector: testAssemblies
testAssemblyVer2: |
D:\a\1\s\Testapp\bin\Debug\net6.0-windows\Testapp.dll
testRunTitle: Warmup
testFiltercriteria: 'TestCategory=TestappE2EWarmup'
I have also tried a few variations, thus
testAssemblyVer2: |
D:\a\1\s\Testapp\Testapp.csproj
and
testFiltercriteria: 'Name=TestWarmup'
What we are getting at the moment is this
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220626-01 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.
vstest.console.exe "D:\a\1\s\Testapp\bin\Debug\net6.0-windows\Testapp.dll"
/TestCaseFilter:"Category=TestappE2EWarmup"
/Settings:"D:\a\_temp\qhskojhglda.tmp.runsettings"
/Logger:"trx"
/TestAdapterPath:"D:\a\1\s"
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
NUnit Adapter 4.0.0.0: Test execution started
Running selected tests in D:\a\1\s\Testapp\bin\Debug\net6.0-windows\Testapp.dll
NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run
NUnit Adapter 4.0.0.0: Test execution complete
No test matches the given testcase filter `Category=TestappE2EWarmup` in D:\a\1\s\Testapp\bin\Debug\net6.0-windows\Testapp.dll
Now from what I understand the VSTest@2 plugin should be able to find Selenium tests and test adapters such as NUnit as per the screen shot below
This is a .NET6.0 project which is being built to replace a .Net Core 3.1 project. The old core code used to find the tests no problem which the code below.
- task: DotNetCoreCLI@2
displayName: 'Regression tests - public'
inputs:
command: test
projects: '$(regressionTests)'
arguments: '--configuration $(BuildConfiguration) --filter "Category=TestappE2EWarmup"'
Can anyone tell me where I have gone wrong please. Thanks in advance
CodePudding user response:
Thanks for the replies, I now having this working by switching from nunit to mstest in the project
<PackageReference Include="MSTest.TestAdapter" Version="2.2.10" />
<PackageReference Include="MSTest.TestFramework" Version="2.2.10" />
<PackageReference Include="SpecFlow.MsTest" Version="3.9.74" />
Now the tests are being found and they run