CI/CD

Visual testing in CI: making screenshots consistent

Most visual testing pain comes from environment differences between local and CI. Learn how to make your CI pipeline produce reliable, reviewable screenshots.

Join the waitlist

Why CI produces different screenshots

Your tests pass locally but fail in CI. The screenshots look almost identical—but "almost" fails a pixel comparison. This isn't flakiness. It's environmental drift.

Operating system rendering

macOS, Linux, and Windows render fonts differently. Anti-aliasing algorithms vary even between Linux distributions.

GPU and acceleration

Hardware rendering produces different results than software rendering. CI runners often lack GPU access entirely.

Browser versions

Even minor browser updates can change rendering. Your local Chrome may differ from CI's pinned version.

Screen scaling and DPI

Retina displays, HiDPI settings, and viewport scaling affect pixel output. CI environments have their own display configurations.

These differences are real. The pixels genuinely differ between environments. The question is whether those differences matter—and how to eliminate the ones that don't.

Make the environment deterministic

Consistency is the goal. Your CI environment should produce identical screenshots every time, matching exactly what baselines were captured against.

Use containerised browsers

Docker containers provide identical OS, fonts, and browser versions across all environments.

Pin browser versions

Lock to specific browser releases. Auto-updating browsers will eventually break your baselines.

Install consistent fonts

Either use web fonts that load before capture, or install identical system fonts in your CI image.

Disable GPU rendering

Force software rendering for consistent output. It's slower but eliminates hardware variance.

The effort here pays dividends. A stable environment means failures are real issues, not environmental noise. For more, see reducing visual testing flakiness.

Keep pipelines fast

Visual tests are slower than unit tests. Plan your pipeline accordingly:

  • Test selection: Run full visual suite on main branch PRs; smoke tests on feature branches
  • Parallelisation: Distribute tests across multiple workers to reduce wall-clock time
  • Separate stages: Consider running visual tests in a parallel pipeline that doesn't block unit test feedback
  • Nightly full runs: For large suites, run comprehensive visual tests overnight rather than on every commit

The goal is fast feedback for most changes, thorough validation before merge.

How to think about baselines in CI

Baselines should be captured in the same environment they'll be tested against. If CI is your test environment, CI should also be your baseline capture environment.

This means baseline updates happen through CI, not local machines. It's more friction, but it eliminates the "works on my machine" problem for visual tests.

For structured approaches to baseline management, see baseline management best practices.

When to block merges vs report only

Not every visual test needs to block deployment. Consider a tiered approach:

  • Block: Design system components, checkout flows, anything where visual bugs directly impact users or revenue
  • Report: Experimental features, rapidly iterating pages, areas where design is still being explored
  • Nightly audit: Full suite runs that surface drift without blocking daily development

Build confidence gradually. Start in reporting mode, prove stability, then escalate to blocking for critical paths.

Quick checklist

  • Use containerised or pinned browser environments
  • Install consistent fonts in CI images
  • Disable GPU rendering for software consistency
  • Configure fixed viewport and display settings
  • Decide on blocking vs reporting mode
  • Set up baseline update approval workflow
  • Document environment requirements for the team

Related guides

How to reduce visual testing flakinessVisual regression testing fundamentalsBaseline management best practicesPlaywright visual testingDesigner-approved visual testing

Frequently Asked Questions

Why do screenshots differ in CI?
CI environments have different operating systems, font rendering engines, browser versions, and display configurations than local machines. These differences produce legitimate pixel-level variance that visual tests detect.
Do I need Docker for consistent screenshots?
Docker is the most reliable way to ensure identical environments, but it's not strictly required. The key is consistency—whatever environment you choose, make it identical between baseline capture and test runs.
Should visual tests run on every PR?
For stable, fast test suites, yes. For slower or flakier suites, consider running on PRs to main only, with full runs nightly. The goal is feedback at the right time without blocking velocity.
How do we stop flaky failures from blocking deploys?
First, fix the flakiness—it's usually environmental. In the meantime, consider running visual tests in reporting mode (not blocking) until they're stable. Always provide manual override for urgent fixes.
How do teams manage baseline updates safely?
Treat baseline updates like code changes: require review, document intent, and tie updates to specific issues or design decisions. Never auto-approve baselines just because tests failed.
How do we handle font rendering differences?
Use web fonts loaded before capture, or standardise on system fonts available in your CI image. Some teams accept slight font variance with threshold-based comparison, but this risks hiding real issues.
Should we run visual tests on feature branches?
A smoke set of critical tests on feature branches catches obvious regressions early. Save the full visual suite for PRs to main, where the cost of thoroughness is justified.
How do we make visual tests faster in CI?
Parallelise across workers, run only affected tests when possible, cache browser installations, and consider separating visual tests into a non-blocking pipeline stage.

Join the waitlist for CI-friendly visual testing

Get early access