| Category | Trainee | 🌟 | 🌟🌟 | 🌟🌟🌟 |
|---|---|---|---|---|
Website Testing | Can accurately compare a finished web page to a design comp/QA checklist and note the differences. Tests rigorously and thinks of possible usages that others haven’t tried yet. Can test invisible functionality like Google Analytics with coaching. Needs a few passes to catch every bug. | Knows the basics of web development and can guess when an issue is frontend, backend, or content. Knows when issues are likely to do with testing tools versus actually being issues. Knows the overall trends in browser usage and understands the browser mix for any given site. Can intuit when differences between comps and pages are acceptable. Can test invisible functionality like Google Analytics without much coaching. Needs fewer passes to catch bugs. | Knows a good amount about web development and can usually tell what part of the stack a bug is coming from. Can perform a strong QA process on any site without a specific checklist to work from (having a checklist is always better of course). Performs additional testing based on this information. Understands the most common types of bugs well and can articulate why they occur. Can write own QA documentation based on a project’s UX documentation and a demo. Usually finds most issues on the first pass. | Can read and understand code so they know how to best challenge a site’s bug-proofness. Has a strong intuition around what aspects of a site are likely to be the biggest problems. Understands the common types of programming bugs and security vulnerabilities and can test against them specifically. Only needs one pass to catch almost any bug. |
Accessibility Testing | Knows the basic principles of web accessibility and what makes for an accessible website. Can test for keyboard accessibility. | Knows how to use at least one screen reader to test websites. Knows all the common accessibility antipatterns (keyboard traps, labels in images, etc) and can identify them on sites. Can use automated tools like AXE to analyze websites and report back to developers on issues. Can check all WCAG AA 2.1 rules which do not involve looking at actual code. | Can check the full WCAG AA 2.1 checklist including reviewing code for invisible or structural problems. Understands common solutions to accessibility issues. | Knows the nuances of different accessibility rulesets and when to deviate from them. Speaks and writes about accessibility QA on behalf of the company. |
Bug Reporting | Can write a clear and accurate bug report using our template. Can guess when a bug is likely a content issue vs. when it is a code problem and note so in the report. | Rarely logs erroneous issues. Can assign issues directly much of the time because they can tell what is going wrong. Uses the CMS to understand the structure of a site. | Can begin to make suggestions about where in the code a problem might lie. Checks the code for hints when encountering an infrequent/unusual problem. | Can write and assign issues directly to developers with recommended solutions based on the contents of the source code. Can assess independently whether an issue has to do with content, and can write a report directly for clients. |
Automated Testing | Has an interest in using automated tools to test sites. | Understands the different types of automated testing available and is learning about how to use them | Can implement at least one strategy for implementing automated acceptance testing | Can implement multiple strategies for writing acceptance tests and connecting them to our continuous integration tools without needing help from DevOps |
Leadership | Can edit existing QA documentation and handbook procedures when changes are needed | Can write handbook entries about our QA process. Contributes to discussions around which testing frameworks and tools to use. | Can independently decide and codify decisions about how the QA department should operate. Is active on our blog as an expert in QA. | Manages other QA Engineers at different stages in their careers. Is visible as a speaker and writer in the general web dev space contributing their thoughts about how QA should be done. |
Working with Developers | Works efficiently with developers to help them replicate bugs. | Can contribute to conversations around Cantilever dev strategies based on findings in prior QA processes. | Can pair with devs to break down issues by looking at the code. | Works with our dev team to modify their practices in response to recurring problems noticed within our source code |