Mutation Testing

Also Known As

TL;DR

Explanation

A mutation testing tool makes small changes (mutations) to the source code — flipping a condition, changing an operator, removing a statement — then runs the test suite. If the tests still pass, the mutation 'survived', meaning no test catches that change. Mutation score = killed mutations / total mutations. A 90% line coverage suite with 40% mutation score has poor actual test quality. Infection is the PHP mutation testing framework.

Diagram

flowchart TD
    SRC[Source Code] --> MT[Mutation Tool<br/>Infection PHP]
    MT -->|Creates| M1[Mutant 1<br/>+ changed to -]
    MT -->|Creates| M2[Mutant 2<br/>if removed]
    MT -->|Creates| M3[Mutant 3<br/>true -> false]
    M1 & M2 & M3 --> SUITE[Test Suite]
    SUITE -->|Tests fail| KILLED[Mutant Killed<br/>good test]
    SUITE -->|Tests pass| SURVIVED[Mutant Survived<br/>test gap found]
style KILLED fill:#238636,color:#fff
style SURVIVED fill:#f85149,color:#fff

Common Misconception

Why It Matters

Common Mistakes

• Running mutation testing on the full codebase at once — start with critical domain logic only; mutation testing is slow.
• Targeting 100% mutation score — some mutations are equivalent (same behaviour, different code) and should be ignored.
• Not interpreting surviving mutants carefully — some indicate test gaps, others are in unreachable code paths.
• Using mutation testing instead of good test design — it is a diagnostic tool, not a substitute for thinking about what to test.

Code Examples

// Test with coverage but no real assertion:
public function testCalculate(): void {
    $calc = new Calculator();
    $result = $calc->add(2, 3); // Line executed — covered
    $this->assertTrue(true);   // No assertion on result — mutation survives!
    // Mutant: return $a - $b; — test still passes
}

// Meaningful assertion — kills the mutant:
public function testCalculate(): void {
    $calc = new Calculator();
    $this->assertSame(5, $calc->add(2, 3));
    $this->assertSame(-1, $calc->add(2, 3, subtract: true));
    // Mutant: return $a - $b; now fails the first assertion
}

References

• ↗ https://infection.github.io/

Tags

Related Terms