Visual Regression Testing

Also Known As

TL;DR

Explanation

Visual regression testing captures screenshots at a pixel or component level and diffs them against approved baselines. Tools: Percy (cloud, integrates with Playwright/Cypress), Chromatic (Storybook integration), BackstopJS (self-hosted), and Playwright's built-in toHaveScreenshot(). Pixel-perfect diffing has high false-positive rates from font rendering and anti-aliasing — threshold-based diffing ignores minor differences. Component-level visual testing (Storybook + Chromatic) is more reliable than full-page screenshot comparison.

Common Misconception

Why It Matters

Common Mistakes

• Pixel-perfect comparison without threshold — minor rendering differences create constant false failures.
• Not updating baselines when intentional changes are made — approved changes are rejected as regressions.
• Running visual tests against dynamic content — timestamps, user avatars, ads cause spurious failures.
• Full-page screenshots for complex pages — component-level testing is more maintainable.

Code Examples

// No visual tests — layout regressions reach production:
// CSS refactor: all tests pass
// Production: every dropdown is 30px off
// Discovered by: customer support ticket 3 days later
// Cost: 3 days * affected users

// Playwright visual regression:
import { test, expect } from '@playwright/test';

test('checkout button renders correctly', async ({ page }) => {
    await page.goto('/checkout');
    await expect(page.locator('.checkout-btn')).toHaveScreenshot(
        'checkout-button.png',
        { threshold: 0.1 } // Allow 10% pixel difference
    );
});
// First run: creates baseline screenshot
// Subsequent runs: diffs against baseline
// CSS regression: test fails with visual diff image

References

• ↗ https://playwright.dev/docs/screenshots

Tags

Related Terms