Screenshot testing is simple to describe and easy to get wrong. You capture an image of a page, capture it again later, and compare the two. If the images differ, something about the page changed.
The idea takes an afternoon to set up. What takes longer is making it trustworthy — because a screenshot is not a stable artefact. The same URL, captured twice from the same machine, can produce two different images for a dozen reasons that have nothing to do with your code. Teams that skip that part end up with a suite that cries wolf, and a team that stops reading it.
This is a practical guide to the part that decides whether screenshot testing survives: getting the same page to produce the same image twice. For the concepts — baselines, what visual regressions look like, when to run comparisons — start with what visual regression testing is.
What a screenshot test actually does
Three steps, whatever tool you use:
- Capture. Load the page in a real browser at a set width and take an image.
- Compare. Diff that image against a stored baseline.
- Decide. A person approves the change or files a bug. Approved images usually become the new baseline.
Everything that goes wrong lives in step 1. Step 2 is arithmetic, and step 3 is only as good as the first two.
Why the same page gives you two different images
The rendering environment
Fonts, GPU, OS and browser version all change how pixels land. Text rendered on macOS is not identical to the same text on Linux, which is why Playwright’s documentation warns that “browser rendering can vary based on the host OS, version, settings, hardware, power source (battery vs. power adapter), headless mode, and other factors” and names its baseline files per platform.
The fix is to capture in one fixed environment and never mix. Locally-generated baselines compared against CI runs is the most common cause of a suite that fails on day one.
Timing
A page is not “done” at load. Web fonts swap in, images decode, lazy-loaded sections appear as the viewport passes them, animations play, skeleton loaders resolve. Capture too early and you photograph a half-built page; capture at a fixed delay and you eventually photograph it in a different state.
What helps, in order of reliability: wait for network idle and for fonts to be ready, disable animations, then add a small delay as a last resort rather than a first one.
Dynamic content
Anything that legitimately differs between loads will be flagged forever: carousels, “latest posts”, relative timestamps, ads, embedded video frames, A/B tests, personalised blocks, randomised ordering. Masking, hiding, freezing or mocking each one is the actual work of screenshot testing, and we wrote it up recipe by recipe in how to avoid false positives.
Page height
Full-page screenshots change height when content changes. A page that grew by one paragraph produces a taller image, and a naive pixel comparison marks everything after that paragraph as different. Some tools compare only the overlapping region, some fail on a size mismatch (BackstopJS has requireSameDimensions for exactly this), and some understand vertical shifts.
Environment drift
If you compare staging against production, content differences will dominate the diff. That is not flake — it is a real difference, just not one about your code. Either sync the content, mock it, or compare the same environment before and after the change instead.
Full page, viewport or element?
Full page is the default for site-wide testing: it catches everything, including things nobody thought to assert. The cost is sensitivity to height changes and lazy-loaded content that only renders while scrolling.
Viewport only (what’s visible without scrolling) is stable and fast, and misses everything below the fold. Useful for a smoke check, not for catching regressions on long marketing pages.
A single element — a header, a pricing table, one card — is the most precise option and the least likely to flake, because everything around it is excluded. This is what component-level tools do, and you can approximate it in most URL-based tools with a CSS selector.
A practical mix for a website: full-page screenshots of one example of each template, plus element screenshots of anything critical and notoriously noisy, like a checkout summary on a page full of recommendations.
One page is several screenshots
The same URL at 375px and at 1440px is two different tests, and most visual regressions are responsive: a grid that collapses, a nav that wraps, a button that leaves the viewport. Our responsive design testing guide covers which widths are worth the screenshots.
Browsers matter too, though less often than widths. Chrome and Safari use different engines, and the same CSS can render differently in each. Testing every page in every browser at every width multiplies fast, so pick a matrix you can afford: all templates at three widths in one browser, plus your highest-value pages in a second browser.
Pages behind a login or a firewall
Most of a real application lives behind a login, so a screenshot tool that can only fetch public URLs covers the marketing site and nothing else. Three things make gated pages testable:
- Cookies — set a session or consent cookie directly so the page renders logged-in, or skip the banner.
- HTTP headers and basic auth — for password-protected staging, and for bypassing a CDN or bot protection that would otherwise serve a challenge page.
- A login step — fill the form before capture when the session can’t be injected.
The same mechanisms handle cookie consent, which is otherwise the single most common false positive on public pages.
Comparing: pixels, thresholds and shifts
Pixel-by-pixel comparison flags every differing pixel. Precise, and noisy about anti-aliasing and font smoothing.
Threshold-based comparison ignores differences below a set percentage — misMatchThreshold in BackstopJS, threshold and maxDiffPixels in Playwright. It quiets anti-aliasing noise, and it also hides small real bugs. A threshold high enough to silence your flake is usually high enough to hide a 4px misalignment.
Shift-aware comparison recognises that a block moved down the page and reports the element that caused the shift, instead of everything below it. This is the difference between a diff you can read and 40 screenshots marked red because a notice bar appeared.
Whatever the method, the output that matters is a review screen a human can scan quickly. If reviewing 200 diffs takes an hour, the suite gets skipped on the day someone is busy — which is every day.
The two-run rule
The single most useful habit in screenshot testing: run the comparison twice with no changes in between.
Everything flagged by that comparison is noise, by definition. Nothing changed. Mask, freeze or mock each item until two consecutive runs come back clean. Only then is the suite worth wiring into a release process, because only then does a red run mean something.
Re-run this check after any significant redesign, and after adding a third-party script. New embeds bring new noise.
Flake or bug?
When a diff appears, the question is which of three things it is:
- Noise — the page differs between loads. Fix the capture, not the code.
- An intended change — you shipped it. Approve it, and it becomes the baseline.
- A regression — nobody meant to change that. File it.
The mistake to avoid is treating a diff you don’t understand as noise. A block of content that moved for no obvious reason is usually a real CSS change with a cause you haven’t found yet. Diffs that look like flake but repeat in the same place deserve a look before the mask goes on.
A checklist before you trust the results
- Screenshots are captured in one fixed environment, not a mix of laptops and CI.
- Animations and transitions are disabled at capture time.
- Fonts and images are loaded before the shutter.
- Consent banners, chat widgets and ads are hidden or handled by cookie.
- Carousels and sliders are frozen to a known slide.
- Dates, “latest” lists and other rotating content are mocked or masked.
- The page list covers one example of each template, at the widths where the layout changes.
- Two consecutive runs with no code change produce no diffs.
- Someone owns the review, and reviewing a full run takes minutes, not an hour.
How this works in Diffy
Diffy is a hosted, URL-based screenshot testing tool: you give it URLs — pasted, imported from a sitemap, or collected from links — and it captures and compares them for you. The things above map to settings rather than code:
- Capture environment is the same cloud browser every time, so there are no per-machine baselines to keep in sync. You choose Chrome or WebKit, at any widths from 240px to 2000px.
- Stabilising a page is handled by masking and hiding elements, setting cookies and HTTP headers, injecting CSS or JavaScript, mocking dynamic text, adding delays and logging in before capture.
- Comparison uses our own algorithm that recognises vertical shifts and points at the element that caused them, with pixel-perfect comparison available when you want every pixel checked.
- Review is a side-by-side screen with per-page and per-breakpoint approval, and shareable links for people without an account.
- Running it can be manual, scheduled, or triggered from CI on paid plans.
The free plan includes 500 screenshots a month with no credit card, which is enough to run the two-run rule on a real site and see how noisy your pages actually are.
Related reading: What is visual regression testing? · How to avoid false positives · Responsive design testing · Stabilising Elementor sites · Screenshots of tabs, menus and sliders · AI screenshot quality checks · The best visual regression testing tools