For .NET Core projects Stryker.NET can be run without any configuration. On .NET Framework projects the solution path is required.
Use a config file
When using Stryker in a team we recomend using a config file. This way you ensure all team members use the same settings to run Stryker. The settings will also be picked up in pipelines. To use a config file create a file called
stryker-config.json in the folder you run Stryker and add a configuration section called stryker-config. Then you can add the options you want to configure to the file.
On .NET Framework projects Stryker needs your
.sln file path.
Stryker.NET needs the path to execute:
When Stryker finds two or more project references inside your test project, it needs to know which project should be mutated. Pass the name of this project using:
Stryker support multiple mutation levels. Each level comes with a specific set of mutations. Each level contains the mutations of the levels below it. By setting the level to
Complete you will get all possible mutations and the best mutation testing experience. This comes at the price of longer runtime, as more mutations have to be tested at higher levels.
The levels are as follows:
- Standard (Default)
|Block (not yet implemented)||Basic|
|String Literals and Constants||Standard|
|Advanced Linq Methods (not yet implemented)||Complete|
|Advanced Regex (not yet implemented)||Complete|
⚠ This parameter is deprecated. This option will be removed in version 1.0. Please submit an issue if you have a use case that requires Dotnet test.
dotnet test, the commandline testrunner and
VsTest, the visual studio testrunner.
VsTest is the default because it offers tight integration with all test frameworks (MsTest, xUnit, NUnit etc).
Dotnet test can be used if VsTest causes issues for some reason. Please also submit an issue with us if you experience difficulties with VsTest.
Some mutations can create endless loops inside your code. To detect and stop these loops, Stryker cancels a unit test run after a set time. Using this parameter you can increase or decrease the time before a test will time out.
During a mutation testrun one or more reporters can be enabled. A reporter will produce some kind of output during or after the mutation testrun.
You can find a list of all available reporters and what output they produce in the reporter docs
Stryker can also be run from the directory containing the project under test. If you pass a list of test projects that reference the project under test, the tests of all projects will be run while testing the mutants.
When running from a test project directory this option is not required.
Logging to console
To gain more insight in what Stryker does during a mutation testrun you can lower your loglevel.
All available loglevels are:
Logging to a file
When creating an issue for Stryker.NET on github you can include a logfile. File logging always uses loglevel
Maximum concurrent test runners
By default Stryker.NET will use as much CPU power as you allow it to use during a mutation testrun. You can lower this setting to lower your CPU usage.
This setting can also be used to disable parallel testing. This can be useful if your test project cannot handle parallel testruns.
your number of logical processors / 2*
* This usually equals your physical processor count
If you want to decide on your own mutation score thresholds, you can configure this with extra parameters.
mutation score > threshold-high:
- Awesome! Your reporters will color this green and happy.
threshold-high > mutation score > threshold-low:
- Warning! Your reporters will display yellow/orange colors, watch out!
threshold-low < mutation score > threshold-break:
- Danger! Your reporters will display red colors, you're in the danger zone now.
threshold-break > mutation score:
- Error! The application will exit with exit code 1.
If you deem some mutations unwanted for your project you can disable mutations.
The mutations of these kinds will be skipped and not be shown in your reports. This can also speed up your performance on large projects. But don't get too exited, skipping mutations doesn't improve your mutation score ;)
⚠ This parameter is deprecated. Use Mutate instead.
If you decide to exclude files for unit testing, you can configure this with the following command:
We recommend to use relative paths. Relative paths are automatically resolved. Absolute paths break easily on different devices. However it is also possible to use absolute paths.
When you want to exclude a large set of files, it is advised to use the stryker configuration file because it is easier to handle multiple files.
To specify which files should be mutated you can use file pattern to in- or excluded files or even only parts of a files. By default all files are included.
The patterns support globbing syntax to allow wildcards.
You can add an
! in front of the pattern to exclude all files that match the pattern.
When only exclude patterns are provided, all files will be included that do not match any exclude pattern. If both, include and exclude patterns, are provided, only the files that match an include pattern but not also an exclude pattern will be included. The order of the patterns is irrelevant.
|Patterns||File||Will be mutated|
To allow more fine grained filtering you can also specify the span of text that should be in- or excluded. A span is defined by the indices of the first character and the last character.
If you would like to ignore some mutations that are passed as method parameters, you can do so by specifying which methods to ignore:
Ignore methods will only affect mutations in directly passed parameters.
You can also ignore constructors by specifying the type and adding the
dotnet stryker -im "['NotImplementedException.ctor']"
Both, method names and constructor names, support wildcards.
dotnet stryker -im "['*Log']" // Ignores all methods ending with log
dotnet stryker -im "['*Exception.ctor']" // Ignores all exception constructors
Config file location
If you want to integrate these settings in your existing settings json, make sure the section is called stryker-config and run stryker with the command
Use coverage info to speed up execution. Possible values are: off, perTest, all, perIsolatedTest.
- off: coverage data is not captured.
- perTest: capture the list of mutants covered by each test. For every mutant that has tests, only the tests that cover a the mutant are tested. Fastest option.
- all: capture the list of mutants covered by each test. Test only these mutants. Non covered mutants are assumed as survivors. Fast option.
- perTestInIsolation: like 'perTest', but running each test in an isolated run. This results in more accurate coverage information for some mutants (see below), at the expense of a longer startup time.
Notes on coverage analysis
- The 'dotnet test' runner only supports
allmode. This is due to dotnet test limitation
- Results are not impacted by coverage analysis. If you identify a suspicious survivor, run Stryker again without coverage analysis and report an issue if this mutant is killed by this run.
- when using
perTestmode, mutants that are executed as part as some static constructor/initializer are run against all tests, as Stryker can not reliably capture coverage for those. This is a consequence of static constructors/initialisers being called only once during tests. This heuristic is not needed when using
perTestInIsolationdue to test being run one by one.
Abort test on fail
Abort unit testrun as soon as any one unit test fails. This can reduce the overall running time.
Enables the diff feature. It makes sure to only mutate changed files. Gets the diff from git by default.
Allows to specify an array of C# files which should be ignored if present in the diff. Any not ignored files will trigger all mutants to be tested because we cannot determine what mutants are affected by these files. This feature is only recommended when you are sure these files will not affect results, or when you are prepared to sacrifice accuracy for performance.
Use glob syntax for wildcards: https://en.wikipedia.org/wiki/Glob_(programming) Example: ['/*Assets.json','/favicon.ico']
Sets the source branch to compare with the current code on file system, used for calculating the difference when --diff is enabled.
This feature works based on file diffs, which means that only changed files will be mutated.
Also note that for changes on test files all mutants covered by tests in that file will be mutated.
EXPERIMENTAL: Dashboard Compare
Enabling the dashboard compare feature saves reports and re-uses the result when a mutant or it's tests are unchanged.
This feature automatically enables the --diff feature.
This feature is experimental. Results can contain slight false postives and false negatives.
When enabling the --dashboard-compare feature you can provide a fallback version. This version will be used to pull a baseline when we cannot find a baseline for your current branch. When we are still unable to provide a baseline we will start a complete testrun to create a complete baseline.
Default: value provided to --git-source or null
Baseline Storage location
Sets the storage location for the baseline used by --dashboard-compare. By default this is set to disk, when the dashboard reporter is enabled this is automatically set to Stryker Dashboard.
Supported storage locations are:
|Disk||Disk||Saves the baseline to the |
|Stryker Dashboard||Dashboard||Saves the baseline to Stryker Dashboard|
|Azure File Storage||AzureFileStorage||Saves the baseline to Azure File Storage|
Configurating Dashboard location
Configuring Azure File Storage
When using Azure File Storage as baseline storage location you are required to provide the following values.
Azure File Storage URL
This is the url to your Azure File Storage. The URL should look something like this:
Providing a subfolder is optional, we store the baseline in a
Shared Access Signature (SAS)
For authentication we support Shared Access Signatures. For more information on how to configure a SAS check the Azure documentation.
The complete configuration would look like this: