Skip to main content

Diffy vs BackstopJS

BackstopJS is a great open source tool that has a lot of similarities with Diffy. We have compiled a list of things that we found different between the too.

 

What Diffy does differently:

 

Time to set up. You can start running Diffy in 5-10 minutes without the need to install anything locally or on your servers. Good for less technical team members like Project Managers. No need to use any CLI tools to run tests.

 

Testing live environments and share results across the team. So you are not dependent on each developer running testing locally. Instead they deploy to some Dev/Stage environment and see results in pull request.

 

Archive.org-type private storage for your website so you can go back in history and see how your site changed over the time.

 

BackstopJS has settings per URL and Diffy does settings globally.

 

Testing public builds. You can add BackstopJS testing during the deployment process, but then you need to take care of where you place artifacts for the testing. For example you could store them in CircleCI https://circleci.com/docs/2.0/artifacts/ but it will take some efforts to make them available for review by your team. With Diffy you have your test results stored and available to your team members right away.

 

Test site as authenticated user. BackstopJS has ability to use custom cookies. But then it will be your job to login user to get authentication cookie and store it before running testing. One of the examples https://medium.com/manati-web-agency/visual-regression-testing-for-drupal-using-backstopjs-61dd09a55a2d.

 

Screenshots memory consumption. Even though BackstopJS recommends that each screenshot takes around 100M from Chrome in reality our tests shown that sometimes it is a much larger amount. Specifically when the page is heavy on images and is very tall. Diffy takes care for this infrastructure challenges.

 

Multiple environments. Backstop has two environments: reference and test. Diffy on the other hand has three (prod, dev, stage). Also it is possible to use any custom environment. So if you need to compare them you do not need to go into settings and modify them.