Skip to main content

One mutation testing HTML report

· 3 min read
Nico Jansen
Nico Jansen
Stryker Team

If you've used Stryker before, you'll know that it produces a gorgeous report you can view in the browser. It works by generating HTML files based on the events raised by Stryker. During development of Stryker4s and Stryker.NET, we realized that producing the same report would be a lot of work.

Instead of a new HTML reporter implementation for each Stryker framework, we've decided to move the logic of presenting the report to your browser. That way, any framework can support an HTML report simply by producing a JSON file.

We call it mutation testing elements and it is implemented using the mature web components suite of features. You can now use it in Stryker, Stryker.NET and Stryker4s

👷‍ Usage​

To enable the new HTML reporter, simply update to the latest Stryker release and enable the HTML reporter in the implementation of your choice.

  • Stryker (since v1.2)
    Enable by adding 'html' to your reporters in your stryker.conf.js file.
  • Stryker4s (since v0.3.1)
    Enable by adding "html" to your reporters in your stryker4s.conf file.
  • Stryker.NET (since v0.9)
    Enable "html" as reporter in your stryker-config.json file.

Upgrading should be painless.

💸 Show me the money​

See two screenshots taken from a mutation testing report of Stryker itself 🎉

Directory

File

Mutation testing elements comes with batteries included:

📊 Calculates and displays the mutation score and other metrics
📁 Group results into directories
👓 Show mutants directly in your source code
😎 Highlight code with embedded highlightjs
🧙‍ Filter mutants based on the outcome
🔗 Deep linking using anchors (uses fragment, so path will not be contaminated)
✨ A nice (yet basic) UI with embedded bootstrap css
🎓 Adheres to custom element best practices

💭 How it works​

Using mutation testing elements is as simple as loading the JavaScript bundle and adding the mutation-test-report-app to your page, pointing the src to a json file that contains the mutation testing results to be displayed.

<body>
  <script src="https://www.unpkg.com/[email protected]/dist/mutation-test-elements.js"></script>
  <mutation-test-report-app src="mutation-report.json">
    Your browser does not support custom elements. Please use a modern browser.
  </mutation-test-report-app>
</body>

Loading from an external source (using src="...") only works if you serve the HTML file from a server. Use property binding to also support opening your HTML report from disk directly.

document.querySelector('mutation-test-report-app').report = {
  /*... report data ...*/
};

The report json should adhere to the mutation testing report json schema, we've designed it with this specific goal in mind.

For more information on how to use it, as well as where to find the JavaScript implementation, please see the readme of mutation testing elements on github. Want to contribute or suggest changes? Feel free to do so in the mutation testing elements mono repo.

💪 Call to action​

Mutation testing elements is here to help out all mutation testing frameworks. We've contacted the guys over at Infection (PHP) and they will support it in the future.

If your mutation testing framework isn't supported, why not open up an issue at their side. Or better yet, implement it in a pull request. Most of them are open source after all. Spread the word 💖 to help us grow 🌳.

🔮 What's next?​

We're planning to upgrade the Stryker dashboard. Right now it already supports showing a mutation score badge. In the future, you will be able to host your entire mutation testing report there. Stay tuned.

Happy mutation testing!