A Hosted Alternative to Nightwatch VRT — No WebDriver Tests to Write

Nightwatch is an end-to-end testing framework built on WebDriver — you write JavaScript tests that drive a real browser, and @nightwatch/vrt adds one assertion type to that toolkit: does this element still look like its baseline. Diffy is a dedicated visual regression platform — no test framework, no WebDriver session, just pages, screenshots and diffs.

That is a real category difference, not a minor one, and it shapes almost everything below. This page compares them anyway, because plenty of teams are deciding between “add a screenshot assertion to the Nightwatch suite we already have” and “stand up a dedicated visual testing tool,” and that is a legitimate choice with real trade-offs on both sides.

What kind of tool each one is

A Nightwatch VRT test is a few lines inside a normal Nightwatch test file:

describe('Homepage', function () {
  test('should match its baseline', function (browser) {
    browser
      .url('https://example.com')
      .waitForElementVisible('body')
      .assert.screenshotIdenticalToBaseline('body', 'homepage', {
        threshold: 0.01,
      });
  });
});

It runs when you run your Nightwatch suite, through whatever driver you’ve configured — ChromeDriver, GeckoDriver, SafariDriver or EdgeDriver — and it needs nightwatch.conf.js to declare the plugin and its paths:

plugins: ['@nightwatch/vrt'],
vrt: {
  latest_screenshots_path: 'vrt/latest',
  baseline_screenshots_path: 'vrt/baseline',
  diff_screenshots_path: 'vrt/diff',
  threshold: 0.0,
},

There is no dashboard, no hosted project, nothing to sign up for. The trade-off is that everything — every page you want tracked, every wait condition, every future change to the list — is a code change reviewed like any other test file.

Diffy has no test files. You create a project in a browser tab, paste a URL list, set breakpoints, and Diffy captures and compares on a schedule or on demand. The trade-off runs the other way: you gain a UI a non-developer can use, and you give up the ability to script a browser session.

Where the screenshots come from

Nightwatch VRT’s capture unit is an element, selected by CSS selector — that is the whole design of screenshotIdenticalToBaseline(selector, fileName, settings, message). You screenshot a .pricing-card, a nav, a .hero. Passing 'body' approximates a full page, but it is still one DOM node rendered at whatever size the WebDriver window happens to be — there’s no dedicated full-page stitching or tall-viewport capture behind it.

Diffy’s capture unit is the page. It opens the URL in a very tall browser window and captures the whole thing in one pass, at every breakpoint you’ve configured, without you naming a single selector.

Neither is strictly better — they’re built for different jobs. Selector-based capture is exactly right for snapshotting one component in isolation inside a larger functional test. Page-based capture is exactly right for “did anything on this page change,” across a whole site, without writing a selector per page.

Comparison that understands vertical shifts

This is the part that changes day-to-day review, and it’s worth being precise about.

Nightwatch VRT uses JIMP for pixel-by-pixel comparison, with a threshold from 0 to 1 controlling tolerance. That’s correct default behavior with a well-known consequence: add one line to a header, everything below it moves down, and every one of those pixels genuinely differs. The whole element lights up red.

Diffy runs a custom algorithm alongside pixel-perfect comparison that recognizes vertical shifts. The same change highlights the header and leaves the rest of the page alone. You can switch to strict pixel-perfect per comparison, or set it as the project default, and tune sensitivity.

Interaction testing: screenshotting mid-flow

This is Nightwatch VRT’s real strength, and it deserves its own section instead of a bullet point.

Because a VRT assertion lives inside a full WebDriver test, you can drive the browser through a real interaction and screenshot the result — a dropdown open, a validation error showing, a modal mid-animation:

browser
  .url('https://example.com/pricing')
  .click('button.open-plan-details')
  .waitForElementVisible('.plan-modal')
  .assert.screenshotIdenticalToBaseline('.plan-modal', 'pricing-modal-open');

Diffy has no equivalent. It can inject custom JavaScript before a capture and it has login presets for Drupal, WordPress and Netlify, but it captures a page state — it does not drive a multi-step session of clicks, waits and typed input. If the thing you need to test is what a page looks like after a user does something, Nightwatch is testing it directly and Diffy is not built for that job.

Real browsers via WebDriver

Nightwatch drives actual browsers through their real drivers — GeckoDriver for Firefox, SafariDriver for Safari, EdgeDriver for Edge, ChromeDriver for Chrome. That’s genuine engine coverage, not emulation.

Diffy renders with Chrome or WebKit. That covers the two engines behind the large majority of real-world traffic, but there is no real Firefox and no real Safari behind it — WebKit approximates Safari’s rendering, it is not Safari.

If a bug has ever shown up only in Firefox for you, this is the section that matters.

Keeping real pages quiet

Most of the effort in visual regression testing isn’t capturing screenshots, it’s stopping them from reporting changes nobody cares about. Nightwatch VRT’s documentation doesn’t describe built-in features for this — masking, content mocking and login handling aren’t part of the plugin. You’d write them yourself, as ordinary WebDriver commands in a before hook, which Nightwatch is entirely capable of — it’s just code you own rather than a setting you flip.

Diffy has these as project settings:

	Nightwatch VRT	Diffy
Hide an element	Not built in — DOM manipulation in a test hook	Mask — paints a visible rectangle, so reviewers can see what was hidden
Remove an element	Not built in — DOM manipulation in a test hook	Removes the node from the DOM
Wait before capture	waitForElementVisible and similar WebDriver commands	Delay setting, plus scroll-to-bottom
Dynamic text	Not built in — stub the data in your test setup	Mock content — replace selector contents with placeholder text
Log in first	WebDriver commands to fill and submit a login form	Presets for Drupal, WordPress and Netlify, plus custom
Custom CSS / JS	Any WebDriver/JS execution the test needs	Fields in project settings
Cookies, headers, User-Agent	WebDriver session capabilities	Fields in project settings
Retina, dark mode	Custom driver capabilities	Checkboxes

Mocking dynamic content. A news listing or a “recently viewed” block re-renders differently on every run against a live CMS. In Nightwatch this is a test-setup problem you solve with fixtures or stubbed data. Diffy takes a list of selectors and replaces their contents with placeholder text, so the listing stops reporting a change every time an article publishes.

Logging in. In Nightwatch, this is normal test code — navigate, fill the form, submit, wait for redirect. In Diffy it’s a preset and two fields.

Reviewing results

Nightwatch VRT writes a report to a local vrt-report folder after each run — baseline, latest capture and a diff image with unmatched pixels marked red, plus a diff percentage. It’s a folder of static files on whichever machine or CI runner produced it. Getting it in front of a non-developer means hosting it somewhere and keeping the history yourself.

Diffy keeps every screenshot and comparison in the cloud:

• Thumbnail view, with approval directly from thumbnails
• The same change grouped across every screenshot it appears in, so you approve it once
• Approve an individual change, a whole screenshot, or a page across all breakpoints
• Before/after slider and keyboard shortcuts for moving through a large set
• Shareable links that work for people without a Diffy account
• Six months of history on paid plans; one month on the free plan

Environments and continuous monitoring

Nightwatch VRT has no concept of environments or scheduling, and that’s consistent with what it is: an assertion that runs when your test suite runs. If you want to compare production against staging, you write a test that visits both URLs and compare the captures yourself. If you want to catch a change nobody deployed — an expired plugin, a third-party script update, a CDN swap — you need to schedule the suite yourself and build the alerting around it.

Diffy models environments — production, staging, development, plus custom ones — as first-class, with pages matched by path against a base URL. That enables scheduled monitoring: Diffy can run daily or weekly and compare an environment against itself over time, with notifications by email, Slack or webhook and configurable thresholds. This is the clearest case where “dedicated platform” and “assertion in a test suite” produce genuinely different capabilities, not just different setup effort.

CI/CD and integrations

Nightwatch integrates by being a test suite: install it, run it in CI, handle the exit code. Its docs cover Jenkins, GitHub Actions, Bamboo, Azure Pipelines, CircleCI and GitLab CI — but that’s Nightwatch’s general CI guidance, not anything specific to VRT results. The report is still a folder your pipeline has to publish somewhere if anyone besides the CI log needs to see it.

Diffy provides:

• A CLI for CI/CD, with worked examples for Pantheon, Tugboat, Platform.sh, CircleCI, GitLab and GitHub Actions
• A GitHub Action that posts results back to the pull request
• Zapier, so a diff can open a ticket in your tracker
• Figma — compare a built page against the design export
• Screenshot upload from Playwright or any other functional testing framework, using Diffy purely as the comparison and review layer

Worth noting: Diffy’s Playwright-upload path means you can keep Nightwatch (or any other framework) for functional and interaction testing, and still send screenshots to Diffy for the parts of visual review — grouping, prioritized diffing, hosted history — that a local plugin doesn’t do. The two tools aren’t strictly either/or.

Driving Diffy from Claude

Diffy publishes a Claude plugin that runs visual testing from a conversation rather than a dashboard or a test file. It installs in two lines:

/plugin marketplace add diffywebsite/diffy-skills
/plugin install diffy@diffy

That gives you skills under the /diffy: namespace covering project setup, capture (locally through the screenshot-worker container, or remotely on Diffy’s servers), comparison, and end-to-end runs that return a percentage changed, a per-page table and a JUnit report. The plugin is MIT licensed and developed in the open at github.com/DiffyWebsite/diffy-skills.

Where Nightwatch VRT is the better fit

We would rather you picked the right tool than the one that emails you.

• You need to test an interaction, not a page load. A dropdown open, a form mid-validation, a modal after a click — Nightwatch screenshots exactly the state you drove it to. Diffy captures pages, not interaction sequences.
• You need real Firefox or real Safari. WebDriver gives you the actual browser. Diffy renders with Chrome or WebKit engines.
• You already have a Nightwatch e2e suite. Adding a VRT assertion to an existing test is one line and zero new tools to adopt.
• You need zero licence cost at any volume. MIT, no per-screenshot fee, ever.
• Nothing may leave your infrastructure. Nightwatch runs entirely in your own CI. Diffy’s local worker keeps capture local but still uploads to Diffy’s cloud for comparison.
• You want component-level snapshots inside a broader test. Selector-based capture is native to Nightwatch VRT; Diffy works at the page level.
• Your baselines should live as files in your own repo. No external service, no account, no vendor dependency for something as basic as “does this still look right.”

Project status, as of this writing

@nightwatch/vrt is MIT licensed and installable today. Its most recent release on npm is 3.5.2, published in May 2024, requiring Node 16 or above. Nightwatch itself is considerably more active — the core framework’s latest release, 3.16.0, shipped in May 2026 — and has been maintained by BrowserStack since 2021, with the project dating back to 2014.

That gap is worth noting rather than glossing over: the framework you’d be building on is healthy, but the specific plugin this comparison is about hasn’t shipped a release in over two years as of this writing. Check the nightwatch-vrt repository directly before you commit a project’s visual testing to it.

Nightwatch and @nightwatch/vrt version, release date, licence and dependency facts verified against the npm registry and the Nightwatch documentation in August 2026.

Frequently asked questions

Is @nightwatch/vrt still maintained?

The plugin is MIT licensed and installable today. Its most recent release on npm is 3.5.2, published in May 2024, maintained by a team that includes several BrowserStack engineers. Nightwatch itself — the parent framework — is far more active, with 3.16.0 shipping in May 2026, so the core project is healthy even though the VRT plugin specifically has gone quiet.

Does Nightwatch VRT capture full-page screenshots?

Not directly. The primitive is screenshotIdenticalToBaseline(selector, ...), which screenshots one DOM element. Passing 'body' as the selector gets close to a full page, but it’s still an element capture — not a dedicated full-page or full-viewport mode the way Diffy or BackstopJS have.

Can I use Nightwatch VRT for free?

Yes. It’s open source under the MIT license with no fee at any volume. The cost is your own CI compute and the engineering time to write and maintain the test files.

Can Diffy click, log in, or open a modal before taking a screenshot?

Partially. Diffy can inject custom JavaScript before capture and has one-click login presets for Drupal, WordPress and Netlify. It can’t script an arbitrary WebDriver interaction — click this, wait for that, then type — the way a Nightwatch test can, because Diffy captures a page rather than driving a multi-step browser session.

Why does Nightwatch VRT flag a whole page when only the header changed?

Nightwatch VRT compares images pixel by pixel using JIMP, with a threshold setting for tolerance. Add a line to a header and everything below it shifts down, so every pixel below it genuinely differs and the diff lights up. Diffy runs a custom algorithm alongside pixel-perfect comparison that recognizes vertical shifts and highlights only the element that actually changed.

Try it

Create an account, paste your URLs, and run your first comparison — the free plan covers 500 screenshots a month and there’s nothing to uninstall if it’s not for you.

If your team is choosing between a hosted platform and a config-file-based tool more broadly, our BackstopJS comparison covers that trade-off directly, and what visual regression testing is covers the basics.