Ghost Inspector

Ghost Inspector

Purpose
Automated Regression Testing
Type
Role-Specific
Status
Active

Overview

Ghost Inspector is a (mostly) no-code approach to automated frontend testing. It allows us to automatically regression test site features on a regular, frequent basis. It integrates well with Slack to notify of test results, and Gitlab as part of a CD/CI strategy.

Rules for Ghost Inspector

✅ Do…

  • Use GI for automated frontend testing and screenshot comparison testing
  • Create a GI test for anything we’d want to test every time we deploy, merge, or even just to make sure it works every so often.
  • Add Ghost Inspector tests during any QA review: If we’re testing it now, we probably want to test it later.

🚫 Don’t…

  • Use GI for accessibility testing. It is not up to our standards for that.

Our Approach to Ghost Inspector

Designing Tests

For best results, a test should be concrete, as small as possible and have a defined answer: if we want to test a navigation menu, we don’t write one test to check everything on the menu. We write multiple tests to check “navigation menu expands on hover”, “search directs to appropriate search result URL”, “contact modal opens and closes”, and so on.

When writing tests, focus on making them modular and able to be used on multiple URLs. Don’t write a test hard-coded for cantilever.co - write a test of steps that we can later choose to use on cantilever.co (or a staging subdomain, or another temporary build URL, or on a local environment, etc).

Client Accounts

Each client ideally has their own plan with Ghost Inspector. Cantilever is invited to the plan to administer it. For smoke tests and the vast majority of clients, the smallest plan (10,000 tests/month) is adequate.

Bipartisan Policy Center purchased their own organization plan.
Bipartisan Policy Center purchased their own organization plan.
Cantilever is added to the organization as an administrator.
Cantilever is added to the organization as an administrator.

Test Construction & Organization

Cantilever builds tests in Ghost Inspector as modules first: these tests are meant to be URL-agnostic and not ran on their own. When we want to run a test, we import the module into a new test and suite, and run that instead. This gives a few benefits:

  1. We only have to build the test once, then can run it at any number of URLs (production vs staging vs merge branch preview, etc).
  2. We have a singular source of truth as far as editing the test: we don’t have to check every suite and edit every copy of the test, we just edit the module.
  3. Because we’re importing the test into suites, and suites have their own settings, we can run the same test at various schedules as-needed.

We organize test suites based on whether it’s a suite for modules, or a suite for running tests.

  • Modules are built by page, region, or general goal of the tests. Examples being:
    • Suite of modules for testing header and footer functionality.
    • Suite of modules for testing a specific page, such as a Home Page or contact page.
    • A generic category that covers testing things that involve multiple pages.
  • Suites for running test are built based on the purpose of the testing suite itself. Modules are imported based on the suite’s need. Examples being:
    • A daily smoke test of a production environment that imports high-priority project-critical regression tests.
    • A daily screenshot comparison of a staging environment. No modules are necessary here: simply build empty tests for the URLs desired.
    • An irregular, twice-weekly test of production environment for lower-priority or less urgent regression tests.
This organization has two folders. The first folder contains three suites of modules, for different purposes or page regions. The second folder contains the actual test suites to execute. These suites import from the module suits as appropriate for the nature of the test run.
This organization has two folders. The first folder contains three suites of modules, for different purposes or page regions. The second folder contains the actual test suites to execute. These suites import from the module suits as appropriate for the nature of the test run.

Working with other web services

Ghost Inspector provides a list of IP Addresses and has a distinct user agent. Services such as Cloudflare, and analytics services, should be configured to accommodate Ghost Inspector - we can’t have tests being blocked by Cloudflare and we don’t want automated tests skewing analytics.

For Cloudflare, the setting needed is Security → WAF → Tools → IP Access Rules. Add Ghost Inspector’s IPs and Allow for This Website.

Guides

Create a Ghost Inspector Test