Skip to main content

Pantheon Quicksilver Quick Start

 

Setup summary

Pantheon has ability of automating tasks that run after deployment. In this tutorial we demonstrate how to set up running visual regression testing after deployment to Test environment.

Summary of setup:

  • Create an authentication key and project in Diffy
  • Set up key and script in your Pantheon’s git repo
  • Run deployment and ensure visual tests

 

Set up Diffy keys and project

First of all you will need to create an account with Diffy.

Create a key with Diffy

Create API Key

After creating a key lets create a project.

Create a Project in Diffy by providing Live and Test URLs

Diffy Create Project

If you are using Trial account that is limited to 10 pages you might see a warning:

“We scanned site and found 29 internal pages, but you have limit of 10. So not all pages listed below.” but that is totally OK. We can start with covering only 10 pages. We would be able to change that easily in the future.

Next notice the ID of the project.

Project ID from URL

After creating a project Diffy will start comparing your Prod environment with Staging. As we haven’t done any deployments yet it should not find any differences. But in my case screenshots were taken from Staging environment too fast and not all images were loaded. To adjust that you can set a delay in project’s settings.

Project settings Delay 10 seconds

 

After setting up a delay I rerun comparison just to ensure no changes are found as sites are identical at this point.

No changes after setting up delay

 

Now we can switch to terminal as we will need to provide credentials to quicksilver so it would be able authenticate with Diffy to trigger our testing.

For that we will need to create secrets.js file and upload it to our site using sftp. So anywhere on your computer run:

echo '{"token": "a9f8a8086c47a52fd000062756f40bc4", "project_id" : "359"}' > secrets.json

And then we need to connect by sftp to Pantheon hosting and upload this file to live environment.

To find sftp credentials go to Pantheon’s dashboard, switch to Live tab and open Connection Info.

sftp -o Port=2222 live.56fe4000-47bc-4273-ab23-3456346dccbc@appserver.live.56fe4929-47bc-4273-ab23-3456346dccbc.drush.in

sftp> cd files
sftp> cd private
sftp> put secrets.json
sftp> quit

So now we have all the configuration on live site.

Last piece would be to set up Quicksilver. 

For that create a folder /private/scripts in your code repository. Then copy a file diffyVisualregression.php from https://github.com/pantheon-systems/quicksilver-examples/tree/master/diffy_visualregression.

Last edit is to let Quicksilver know to run that script on deployment. So edit your pantheon.yml file from code repository’s root folder and add a step:

workflows:
  deploy:
    after:
      - type: webphp
        description: Do a visual regression test with Diffy.website
        script: private/scripts/DiffyVisualregression.php

Now we can commit and push the code. It will automatically get deployed to Dev environment.

Now is the moment of truth -- lets run deployment on Test (staging) environment. For that go to your dashboard and run deployment.

Deploy to test environment Pantheon

 

In Diffy you should see screenshots and diffs started.

Diffy jobs started

 

After all the jobs completed -- you will receive a notification and would be able to check the differences.

Congratulations!

 

Next steps

As next steps you could set up different notifications. For example to visually compare test and develoment environments on deployment to development. Also you can set up a slack notification once visual testing is completed. Let me know if you come up with any other ideas in comments.

 

Troubleshooting

If something doesn’t work for you. Best way to debug the deployment script is via Terminus (replace diffy with your sitename)

./terminus workflow:watch diffy

This will show you execution log of the deployment script diffyVisualregression.php. So you could add different “print_r()” to debug what is wrong.

 

Youtube recorded demo

Welcome to watch a video demo.

Please let us know how it works for you.