Mutation Testing
debt(d5/e3/b3/t7)
Closest to 'specialist tool catches it' (d5). The detection_hints.tools list includes Infection PHP, a specialist mutation testing tool — it's not caught by the compiler or a default linter, but a dedicated tool (Infection) can surface surviving mutants when explicitly run. PHPUnit and Pest alone won't flag the problem; Infection must be deliberately integrated into the pipeline.
Closest to 'simple parameterised fix' (e3). The quick_fix describes running Infection PHP on the unit test suite and adding assertions that verify actual output. This is typically a small, localised fix per surviving mutant — adding or strengthening assertions in a test file — but it may touch multiple test cases across one component, placing it at e3 rather than a trivial one-liner.
Closest to 'localised tax' (b3). Mutation testing is a diagnostic/quality tool rather than a load-bearing architectural choice. It applies to specific test suites (web, cli, queue-worker contexts) and its cost is borne mainly by the team maintaining the test suite. Common mistakes note it should be scoped to critical domain logic, limiting its reach. It doesn't shape the whole codebase architecture, keeping burden at b3.
Closest to 'serious trap' (t7). The misconception field directly states the trap: developers believe 100% code coverage means tests are thorough, but mutation testing proves otherwise. This contradicts the widely held mental model that coverage metrics are a reliable proxy for test quality — a belief reinforced by tooling defaults (coverage reports) across the industry. This is a well-documented gotcha that contradicts how a similar concept (code coverage) is understood, warranting t7.
Also Known As
TL;DR
Explanation
A mutation testing tool makes small changes (mutations) to the source code — flipping a condition, changing an operator, removing a statement — then runs the test suite. If the tests still pass, the mutation 'survived', meaning no test catches that change. Mutation score = killed mutations / total mutations. A 90% line coverage suite with 40% mutation score has poor actual test quality. Infection is the PHP mutation testing framework.
Diagram
flowchart TD
SRC[Source Code] --> MT[Mutation Tool<br/>Infection PHP]
MT -->|Creates| M1[Mutant 1<br/>+ changed to -]
MT -->|Creates| M2[Mutant 2<br/>if removed]
MT -->|Creates| M3[Mutant 3<br/>true -> false]
M1 & M2 & M3 --> SUITE[Test Suite]
SUITE -->|Tests fail| KILLED[Mutant Killed<br/>good test]
SUITE -->|Tests pass| SURVIVED[Mutant Survived<br/>test gap found]
style KILLED fill:#238636,color:#fff
style SURVIVED fill:#f85149,color:#fff
Common Misconception
Why It Matters
Common Mistakes
- Running mutation testing on the full codebase at once — start with critical domain logic only; mutation testing is slow.
- Targeting 100% mutation score — some mutations are equivalent (same behaviour, different code) and should be ignored.
- Not interpreting surviving mutants carefully — some indicate test gaps, others are in unreachable code paths.
- Using mutation testing instead of good test design — it is a diagnostic tool, not a substitute for thinking about what to test.
Code Examples
// Test with coverage but no real assertion:
public function testCalculate(): void {
$calc = new Calculator();
$result = $calc->add(2, 3); // Line executed — covered
$this->assertTrue(true); // No assertion on result — mutation survives!
// Mutant: return $a - $b; — test still passes
}// Meaningful assertion — kills the mutant:
public function testCalculate(): void {
$calc = new Calculator();
$this->assertSame(5, $calc->add(2, 3));
$this->assertSame(-1, $calc->add(2, 3, subtract: true));
// Mutant: return $a - $b; now fails the first assertion
}