Overview

Ghost Inspector is a (mostly) no-code approach to automated frontend testing. It allows us to automatically regression test site features on a regular, frequent basis. It integrates well with Slack to notify of test results, and Gitlab as part of a CD/CI strategy.

Rules for Ghost Inspector

✅ Do…

• Use GI for automated frontend testing and screenshot comparison testing when content doesn’t constantly change
• Create a GI test for anything we’d want to test every time we deploy, merge, or even just to make sure it works every so often.
• Add Ghost Inspector tests during any QA review: If we’re testing it now, we probably want to test it later.

🚫 Don’t…

• Use GI as the only tool for accessibility testing. Ghost Inspector’s testing is based on axe-core, which covers a lot, but manual testing is needed for full coverage.

Ideal Implementation

The ideal implementation for any site should meet this list. More can be done, but this is considered the minimum for a “complete” usage. Diverge from it only in exceptional cases.

• [ ] The Ghost Inspector dashboard has a folder for the site.
• [ ] Suites:
  • [ ] “Modules” suite with import-only tests.
  • [ ] Testing suites that import those modules to be ran:
    • [ ] “Testing” suites for Production/Staging/Other Environments, for functionality testing.
    • [ ] “Visual” suites for screenshot comparisons.
  • [ ] Suites are either set on a schedule, and/or use a webhook to run as part of CD/CI integration.
  • [ ] Suites have notifications set:
    • [ ] Notifications are sent to Slack to at least two channels: #ghost-inspector-alerts, and any appropriate channels for the client/site project.
• [ ] Naming Conventions:
  • [ ] Folder is labeled with client code and website’s full name, such as “[CAN] Cantilever”.
  • [ ] Suites are given the client code and site name, such as “[CAN] Cantilever, Modules”
  • [ ] For testing suites, describe the suite’s purpose and environment, such as “[CAN] Cantilever, Testing, Production” or “[CAN] Cantilever, Visual, Staging”.
• [ ] Internally consistent variable naming
  • [ ] Variable for the base URL should be baseURL (case sensitive). The URL should not include a trailing slash. Example: base URL variable for testing cantilever.co should be https://cantilever.co (no trailing slash) for the variable {{baseURL}}. Visiting /contact for the page would be constructed as {{baseURL}}/contact
  • [ ] If a test uses a form, form details use the organization-wide variables and email convention defined in the Forms section of the handbook.
• [ ] If a suite or test does not meet our existing conventions, it should have a comment detailing what conventions it breaks and why meeting the convention isn’t possible or practical.

Our Approach to Ghost Inspector

Designing Tests

For best results, a test should be concrete, as small as possible and have a defined answer: if we want to test a navigation menu, we don’t write one test to check everything on the menu. We write multiple tests to check “navigation menu expands on hover”, “search directs to appropriate search result URL”, “contact modal opens and closes”, and so on.

When writing tests, focus on making them modular and able to be used on multiple URLs. Don’t write a test hard-coded for cantilever.co - write a test of steps that we can later choose to use on cantilever.co (or a staging subdomain, or another temporary build URL, or on a local environment, etc).

Naming Conventions

• Folder is labeled with client code and site name: [CAN] Cantilever. All suites related to the site go in their folder.
  • Each site should have one folder. If a client has multiple sites, make a folder for each site: “[CAN] Cantilever” and “[CAN] Sherbert Fan Club”.
• Suites are given the client code, site name, and purpose.
  • For ordering, think Who, What, Where. Examples follow.
  • [CAN] Cantilever, Testing, Production
    • Who: Cantilever
    • What: Testing
    • Where: Production
  • [CAN] Cantilever, Visual, Staging, EU Site
    • Who: Cantilever
    • What: Visual (comparison tests)
    • Where: Staging environment, EU subsite.
  • [CAN] Cantilever, Modules
    • Who: Cantilever
    • What: Modules
    • Where: Nowhere specific, they’re modules.

Test Construction & Organization

Cantilever builds tests in Ghost Inspector as modules first: these tests are meant to be URL-agnostic and not ran on their own. When we want to run a test, we import the module into a new test and suite, and run that instead. This gives a few benefits: