Pantheon Quicksilver Quick Start
Setup summary
Pantheon has the ability to automate tasks that run after deployment. In this tutorial, we demonstrate how to set up running visual regression testing after deployment to the Test environment.
Summary of setup:
- Create an authentication key and project in Diffy
- Set up key and script in your Pantheon’s git repo
- Run deployment and ensure visual tests
Set up Diffy keys and project
First of all, you will need to create an account with Diffy.
Create an API key with Diffy (under the main navigation https://app.diffy.website/#/
After creating a key let's create a project.
Create a Project in Diffy by providing Live and Test URLs
If you are using a Trial account that is limited to 10 pages you might see a warning:
“We scanned the site and found 29 internal pages, but you have a limit of 10. So not all pages listed below.” but that is totally OK. We can start by covering only 10 pages. We would be able to change that easily in the future.
Next, notice the ID of the project.
After creating a project Diffy will start comparing your Prod environment with Staging. As we haven’t done any deployments yet it should not find any differences. But in my case screenshots were taken from the Staging environment too fast and not all images were loaded. To adjust that you can set a delay in the project’s settings.
After setting up a delay I rerun the comparison just to ensure no changes are found as sites are identical at this point.
Now we can switch to the terminal as we will need to provide credentials to quicksilver so it would be able to authenticate with Diffy to trigger our testing.
For that, we will need to create a secrets.js file and upload it to our site using sftp. So anywhere on your computer run (token is the Diffy's API key we just generated):
echo '{"token": "a9f8a8086c47a52fd000062756f40bc4", "project_id" : "359"}' > secrets.json
And then we need to connect by sftp to Pantheon hosting and upload this file to the live environment.
To find sftp credentials go to Pantheon’s dashboard, switch to Live tab, and open Connection Info.
sftp -o Port=2222 live.56fe4000-47bc-4273-ab23-3456346dccbc@appserver.live.56fe4929-47bc-4273-ab23-3456346dccbc.drush.in
sftp> cd files
sftp> cd private
sftp> put secrets.json
sftp> quit
So now we have all the configurations on the live site. Please also sync the files between your environments so secret file is populated to other environments as well.
The last piece would be to set up Quicksilver.
For that create a folder `/private/scripts` in your web docroot which may be in `web` so that would be `/web/private/scripts`. See Pantheon's documentation for more details. Then copy a file diffyVisualregression.php from https://github.com/pantheon-systems/quicksilver-examples/tree/master/diffy_visualregression.
The last edit is to let Quicksilver know to run that script on deployment. To edit your pantheon.yml file from the code repository’s root folder and add a step:
workflows:
deploy:
after:
- type: webphp
description: Do a visual regression test with Diffy.website
script: private/scripts/diffyVisualregression.php
Now we can commit and push the code. It will automatically get deployed to the Dev environment.
Now is the moment of truth -- let's run deployment on the Test (staging) environment. For that go to your dashboard and run deployment.
In Diffy you should see screenshots and diffs started.
After all the jobs are completed -- you will receive a notification and would be able to check the differences.
Congratulations!
Next steps
As the next steps, you could set up different notifications. For example to visually compare test and development environments on deployment to development. Also, you can set up a slack notification once visual testing is completed. Let me know if you come up with any other ideas in the comments.
Troubleshooting
If something doesn’t work for you. The best way to debug the deployment script is via Terminus (replace diffy with your sitename)
./terminus workflow:watch diffy
This will show you the execution log of the deployment script diffyVisualregression.php. So you could add different “print_r()” to debug what is wrong.
Alternatively you can debug script execution via terminus
./terminus remote:drush site.environment -- php:script diffyVisualregression.php --script-path=/code/web/private/scripts
Youtube recorded demo
Welcome to watch a video demo.
Please let us know how it works for you.