- Testing
- Quality
Your test suite is green. Every test passes. But does that prove your tests are any good — or only that they run? A passing suite can still miss real bugs. Mutation testing is how you find out.
The idea
A mutation testing tool makes small, deliberate breaks in your code. It might change a + to a -, flip a > to a <, or delete a line. Each broken copy is a mutant. The tool then runs your test suite against each one.
The logic is simple. If you break the code and a test fails, that is good — your tests caught the fault. The mutant is "killed." But if you break the code and every test still passes, you have a problem. The mutant "survived," which means a real fault of that shape would also get past you.
The result is a mutation score: the percentage of mutants your tests killed. It measures how much your tests actually check, not only how many lines they touch.
Why it beats coverage
Line coverage asks "did a test run this line?" You can run a line without checking its result at all. Mutation testing asks the harder question: "would a test notice if this line were wrong?" That is much closer to what you want to know.
The trade-offs
Pros: it finds weak and meaningless tests, it shows gaps that coverage hides, and it pushes you to write assertions that matter.
Cons: it is slow. Each mutant means another full test run, so large projects can take hours. It also produces equivalent mutants — changes that do not alter real behaviour and so can never be killed. These make noise that you must review by hand.
Where to start
Do not run it over everything at once. Point a tool — Stryker, PIT, or mutmut — at one important module. Read the survivors. Each one is a small, honest lesson about a test you thought was protecting you.
Written by base32.