Overview

Use For...

✅ Automated front-end testing: Ghost Inspector tests are slightly more advanced than a user clicking and typing on the page, and can test similar cases.

✅ ”Screenshot comparison tests”: Ghost Inspector can compare screenshots of a site over time and report if the visuals look too different based on a threshold.

Don’t Use For...

⛔️ Accessibility testing - Ghost Inspector has basic accessibility tools but insufficient for our standards

⛔️ Back-end testing or code linting. Ghost Inspector can test that a button activates a widget and displays data: it has no way of knowing how the button’s widget request for data is handled by the server.

Why we use it

Ghost Inspector is a (mostly) no-code approach to automated frontend testing. It allows us to automatically regression test site features on a regular, frequent basis. It integrates well with Slack to notify of test results, and Gitlab as part of a CD/CI strategy.

How we use it

‣

Client Management

Each client ideally has their own plan with Ghost Inspector. Cantilever is invited to the plan to administer it. For smoke tests and the vast majority of clients, the smallest plan (10,000 tests/month) is adequate.

Bipartisan Policy Center purchased their own organization plan.

Cantilever is added to the organization as an administrator.

‣

Test Construction & Organization

For more information, see:

Cantilever builds tests in Ghost Inspector as modules first: these tests are meant to be URL-agnostic and not ran on their own. When we want to run a test, we import the module into a new test and suite, and run that instead. This gives a few benefits:

We only have to build the test once, then can run it at any number of URLs (production vs staging vs merge branch preview, etc).
We have a singular source of truth as far as editing the test: we don’t have to check every suite and edit every copy of the test, we just edit the module.
Because we’re importing the test into suites, and suites have their own settings, we can run the same test at various schedules as-needed.

We organize test suites based on whether it’s a suite for modules, or a suite for running tests.

Modules are built by page, region, or general goal of the tests. Examples being:

Suite of modules for testing header and footer functionality.
Suite of modules for testing a specific page, such as a Home Page or contact page.
A generic category that covers testing things that involve multiple pages.

Suites for running test are built based on the purpose of the testing suite itself. Modules are imported based on the suite’s need. Examples being:

A daily smoke test of a production environment that imports high-priority project-critical regression tests.
A daily screenshot comparison of a staging environment. No modules are necessary here: simply build empty tests for the URLs desired.
An irregular, twice-weekly test of production environment for lower-priority or less urgent regression tests.

This organization has two folders. The first folder contains three suites of modules, for different purposes or page regions. The second folder contains the actual test suites to execute. These suites import from the module suits as appropriate for the nature of the test run.

‣

Working with other web services

Ghost Inspector provides a list of IP Addresses and has a distinct user agent. Services such as Cloudflare, and analytics services, should be configured to accommodate Ghost Inspector - we can’t have tests being blocked by Cloudflare and we don’t want automated tests skewing analytics.

For Cloudflare, the setting needed is Security → WAF → Tools → IP Access Rules. Add Ghost Inspector’s IPs and Allow for This Website.

Golden Rules

If there’s something we’d want to test every time we deploy, merge, or even just to make sure it works every so often, consider a Ghost Inspector test for it.
An excellent time to add a Ghost Inspector test is during any QA review: If we’re testing it now, we probably want to test it later.
For best results, a test should be concrete, as small as possible and have a defined answer: if we want to test a navigation menu, we don’t write one test to check everything on the menu. We write multiple tests to check “navigation menu expands on hover”, “search directs to appropriate search result URL”, “contact modal opens and closes”, and so on.
When writing tests, focus on making them modular and able to be used on multiple URLs. Don’t write a test hard-coded for cantilever.co - write a test of steps that we can later choose to use on cantilever.co (or a staging subdomain, or another temporary build URL, or on a local environment, etc).