Stryker

Test your tests with mutation testing.

Stryker
Stryker

Getting started with Stryker

What is mutation testing?

Bugs, or mutants, are automatically inserted into your production code. Your tests are ran for each mutant. If your tests fail then the mutant is killed. If your tests passed, the mutant survived. The higher the percentage of mutants killed, the more effective your tests are.

It's really that simple.

Still confused? Why not taking a look at our example page and try it out yourself?

But wait, what about code coverage?

Well... code coverage doesn't really tell you everything about the effectiveness of your tests. Think about it, when was the last time you saw a test without an assertion, purely to increase the code coverage?

Imagine a sandwich covered with paste. Code coverage would tell you the bread is 80% covered with paste. Mutation testing, on the other hand, would tell you it is actually chocolate paste and not... well... something else.

Meet: Stryker

Sounds complicated? Don't worry, Stryker has your back. It uses one design mentality to implement mutation testing on 3 platforms. It's easy to use and fast to run. Stryker will only mutate your source code, making sure there are no false positives.

An example

Say you're building an online casino. Users can only enter the casino when they're over 18. So you write the following piece of code to check if someone is allowed to use the website:

function isUserOldEnough(user) {
  return user.age >= 18;
}

Stryker will find the return statement and decide to change it in several ways:

/* 1 */ return user.age > 18;
/* 2 */ return user.age < 18;
/* 3 */ return false;
/* 4 */ return true;

We call those modifications mutants. After the mutants have been found, they are applied one by one and your tests are executed against them. If at least one of your tests fail, we say the mutant is killed. That's what we want! If no test fails, it survived. The better your tests, the fewer mutants survive.

Stryker will output the results in various different formats. One of the easiest to read is the clear text reporter:

  Mutant killed: /yourPath/yourFile.js: line 10:27
  Mutator: BinaryOperator
  -                 return user.age >= 18;
  +                 return user.age > 18;

  Mutant survived: /yourPath/yourFile.js: line 10:27
  Mutator: RemoveConditionals
  -                 return user.age >= 18;
  +                 return true;

The clear text reporter will output how exactly your code was modified and which mutator was used. It will then tell us if a mutant was killed, meaning that at least one test failed, or if it survived. The second mutation in this example is marked as a survivor. This means there is probably a test missing that explicitly tests for age lower than 18.

Features

Mutations

Control more than 30 supported mutations, or write your own.

Speed

Use code analysis and parallel test runner processes to speed things up.

Test runner agnostic

Run tests with your favorite test runner.

Open source

Free as in Speech. Maintained by the open source community at GitHub.

Multilingual

Stryker has support for JavaScript & TypeScript, C# and Scala

Clever reports

Use clever reports to sniff out surviving mutants and improve test effectiveness.

Info Support

Stryker is sponsored by Info Support. They provide our awesome swag, host Stryker hackathons and provide time for their developers to work on Stryker and other open source projects. Check them out at opensource.infosupport.com.

For someone who hates mutants... you certainly keep some strange company.

Professor X

Oh, they serve their purpose... as long as they can be controlled.

William Stryker
Need help?