Installing Stryker in pipelines

When running stryker in your pipeline there are some things to take into consideration

Due to the way dotnet core global tools are installed on the system a regular dotnet tool install -g is often not effective in pipelines.

Instead use the --tool-path to install stryker in a local folder or use the project level install of dotnet core 3.0+

Example for installing in azure devops:

displayName: 'Install dotnet-stryker'
command: custom
custom: tool
arguments: install dotnet-stryker --tool-path $(Agent.BuildDirectory)/tools

And then running this locally installed tool:

displayName: 'Run dotnet-stryker'
workingDirectory: <test-project-folder-here>
targetType: 'inline'
pwsh: true
script: $(Agent.BuildDirectory)/tools/dotnet-stryker

Configuring dashboard compare in pull requests

Dashboard compare is very useful when running stryker in pipelines because it cuts down on the total runtime by only testing mutations that have changed compared to for example master The following minimal steps are needed to use dashboard compare

  1. Enable --dashboard-compare
  2. Choose a storage provider (Dashboard for public projects or Azure File Share for private projects)
  3. Set up authentication for the chosen storage provider
  4. Set --dashboard-version to the name of the source branch (usually current branch)
  5. Set --git-source to the name of the target branch (usually master/main or development)
  6. Set any other options needed for your chosen storage provider (see: Configuration)

Example for azure devops with dashboard storage provider:

dotnet stryker --dashboard-compare --baseline-storage-location Dashboard --dashboard-api-key $(Stryker.Dashboard.Api.Key)--dashboard-version $(System.PullRequest.SourceBranch) --git-source $(System.PullRequest.TargetBranch)
dotnet stryker -compare -bsl Dashboard -dk $(Stryker.Dashboard.Api.Key)-version $(System.PullRequest.SourceBranch) -source $(System.PullRequest.TargetBranch)

Azure DevOps Extensions

There are multiple Azure DevOps extensions to enrich the Stryker experience.

Mutation Report Publisher

With this extension you add the mutation report to your build summary so you can view the mutation test result on azure devops.

In YAML pipelines add the following after the test step:

displayName: 'Publish Mutation Test Report'
reportPattern: '**/mutation-report.html'

You now get a new tab on your build pipeline where you can click through the complete mutation testing report generated by your mutation test run. When using Stryker the mutation testing report looks like:

mutation report in a build

To install 'Mutation Report Publisher' or for more information go to the marketplace.

Run Stryker .Net

โš  This is not an official Stryker Mutator extension. This extension is not cross platform and only works on Windows.

This is an Azure Devops extension to run mutation testing with Stryker .Net without having to install Stryker yourself.

In YAML pipelines add the following after the test step:

- task: run-[email protected]
testProject: '*Tests.csproj' #example value
parameters: '--reporters "['html', 'progress']" #example value'

To install 'Run Stryker .Net' or for more information go to the marketplace.