Skip to main content

Welcome to the RoboCoasters 🤖🎢

An introduction to mutation testing

How code coverage of 100% could mean only 60% is tested.

RoboCoasters application

TL;DR​

No time to run the example yourself? Don't worry; we did it for you. Open it right in your browser:

What is this?​

RoboCoasters is a small application to demo mutation testing. It has a fair amount of unit tests. We didn't try our best to write bad tests when we wrote this application. We just focussed on code coverage and didn't practice Test Driven Development. It turns out it's easy to write bad tests or forget a few important test cases. RoboCoasters even has a fairly large bug. Finding it is easy using the mutation report. Why don't you give it a try? 😁

Note: Robo coasters is developed using native web components without a frontend framework. This is done on purpose to keep this example as accessible as possible and to keep the maintenance burden low.

Try it yourself​

  1. Install git
  2. Install nodejs
  3. Open command prompt and clone this repository: 
    git clone https://github.com/stryker-mutator/robo-coasters-example
  4. Change directory into robo-coasters-example and install the dependencies. 
    cd robo-coasters-example
    npm install
  5. Run tests with npm. This will generate a code coverage report. 
    npm test
  6. Review the 100% code coverage score. Open up the code coverage report in the reports/coverage/lcov-report directory.
  7. Run mutation testing with Stryker 
    npm run test:mutation
  8. Review ~60% mutation score. Open up the mutation report in the reports/mutation directory.
  9. Run the website with npm start. Can you find the bug?

Try to install stryker yourself.​

If you want to install stryker yourself, step back in history using git:

git checkout pre-stryker
npm install

After that you can install stryker for yourself:

npm init stryker

Choose the following options in the questionnaire:

  • Are you using one of these frameworks? None/other
  • Which test runner do you want to use? jest
  • Reporters: html, clear-text, progress
  • Which package manager do you want to use?: npm
  • What file type do you want for your config file?: json

After the plugins are installed, open the stryker.conf.json file and make the following change:

{
  "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
  "_comment": "This config was generated using 'stryker init'",
  "packageManager": "npm",
  "reporters": [
    "html",
    "clear-text",
    "progress",
    "dashboard"
  ],
  "testRunner": "jest",
- "coverageAnalysis": "perTest"
+ "coverageAnalysis": "perTest",
+ "testRunnerNodeArgs": ["--experimental-vm-modules"]
}

(this is needed because we're using jest with ECMAScript modules)

After the plugins are installed, try it out:

npx stryker run