Skip to main content
Accessibility.com, LLC
Category: Testing and Evaluation

Automated Testing

Also known as: Test Automation, Automated Accessibility Testing
Simply put

Automated testing is the use of software tools to run predefined checks against a website or application, rather than having a person perform each check by hand. In accessibility work, these tools can quickly scan pages for certain kinds of issues and report potential problems. However, automated tools detect only a portion of accessibility issues, so manual review and assistive technology testing are generally still needed.

Formal definition

Automated testing is a testing approach in which software tools and scripts execute predefined test cases repeatedly and at scale, comparing actual outcomes against expected results without a human running each test manually. Applied to digital accessibility, automated tools programmatically evaluate markup, computed properties, and rendered output against machine-detectable conditions derived from standards such as WCAG success criteria. These tools can reliably surface certain classes of defects (for example, missing alternative text attributes, some color-contrast failures, and specific ARIA misuse), but many WCAG requirements depend on human judgment (such as the meaningfulness of alternative text, logical reading order, or the accuracy of labels) and are not fully verifiable by automation. Consequently, automated testing detects only a subset of potential issues and should be combined with manual inspection and assistive technology testing; passing automated checks does not by itself establish WCAG conformance, an accessible user experience, or legal compliance. This entry is general guidance and not legal advice.

Why it matters

Automated testing is valuable because it lets teams scan large volumes of pages quickly and repeatedly, catching certain classes of defects far faster than manual review alone. For organizations maintaining sites that change frequently, integrating automated checks into a build process or continuous integration pipeline can surface machine-detectable problems (such as missing alternative text attributes or some color-contrast failures) early, before they reach users. This scalability makes automated testing a practical first line of defense in an accessibility program.

The critical caveat is that automated tools detect only a portion of accessibility issues. Many WCAG requirements depend on human judgment, whether alternative text is actually meaningful, whether reading order is logical, or whether labels accurately describe their controls, and these cannot be fully verified by software. Relying on automated results alone can create a false sense of security, because passing automated checks does not by itself establish WCAG conformance, an accessible experience for all users, or immunity from legal claims.

For this reason, automated testing should be treated as one component of a broader evaluation strategy rather than a complete solution. Combining it with manual inspection and testing with assistive technologies gives a far more accurate picture of real-world accessibility. This entry is general guidance and not legal advice; organizations with specific compliance questions should consult qualified legal counsel and monitor current agency rulemaking.

Who it's relevant to

Accessibility Engineers and QA Testers
Automated testing gives engineers and testers a scalable way to catch machine-detectable defects early and consistently, particularly when integrated into build pipelines. They should understand which issue classes automation reliably surfaces and which require manual verification, and plan complementary manual and assistive technology testing accordingly.
UX and Front-End Developers
Developers benefit from running automated checks during the build process to catch markup and property-level issues before they ship. Understanding the limits of automation helps them avoid treating a passing scan as proof that a component is fully accessible, since judgment-dependent criteria still need human review.
Compliance Officers and Program Managers
Those responsible for accessibility programs need to know that automated results represent only a portion of potential issues. Passing automated checks does not by itself establish WCAG conformance or legal compliance, so automated testing should be positioned as one input into a broader, documented evaluation process.
Legal Counsel and Business Leaders
Decision-makers should recognize that automated testing supports, but does not replace, a comprehensive accessibility effort. Because requirements evolve through regulation and case law and automation cannot guarantee an accessible experience for all users, this guidance is not legal advice, and specific compliance questions should be directed to qualified counsel.

Inside Automated Testing

Automated Accessibility Scanning
Software-driven evaluation of digital content against machine-detectable accessibility rules, commonly mapped to WCAG success criteria. It programmatically inspects code and rendered output to flag potential issues.
Rule-Based Detection
Checks that identify programmatically determinable problems, such as missing alternative text attributes, absent form labels, empty links or buttons, and certain color contrast failures that can be measured computationally.
Coverage Scope
Automated tools generally detect only a portion of accessibility barriers. Many WCAG requirements depend on context, meaning, and user interaction that software cannot reliably evaluate, so a passing scan does not indicate full conformance.
Integration Points
Automated testing can be run as browser extensions, command-line tools, or integrated into continuous integration and development pipelines to catch regressions early in the workflow.
Output and Reporting
Results typically include identified issues, the relevant WCAG success criteria referenced, severity indicators, and the location in the code, which teams use to prioritize remediation.

Common questions

Answers to the questions practitioners most commonly ask about Automated Testing.

Can automated testing alone confirm that a website is fully accessible or WCAG conformant?
No. Automated testing detects only a portion of potential accessibility issues, commonly the machine-detectable ones such as missing alternative text attributes, certain color contrast failures, or absent form labels. Many WCAG success criteria require human judgment to evaluate, such as whether alternative text is meaningful, whether reading order is logical, or whether keyboard focus behavior makes sense. Passing automated checks does not by itself establish full WCAG conformance, and manual and assistive technology testing are generally required to reach a reasonable assessment.
Does passing automated accessibility scans guarantee protection from legal claims?
No. Meeting WCAG success criteria or passing automated checks does not guarantee an accessible experience for all users, nor does it guarantee immunity from legal claims. Legal exposure under frameworks such as the ADA is shaped by regulation, agency guidance, and case law, which continue to evolve. Automated results are one input into an accessibility program, not a compliance certification. Organizations should consult qualified legal counsel regarding their specific circumstances; this entry is not legal advice.
How should automated testing be combined with other testing methods?
Automated testing is generally most effective as an early, repeatable layer that catches machine-detectable issues quickly and at scale. It is commonly paired with manual testing, such as keyboard-only navigation and code review, and with testing using assistive technologies like screen readers. Because automated tools cannot evaluate many context-dependent criteria, a combined approach is typically needed to assess conformance and usability more completely.
At what point in the development process should automated testing be applied?
Automated testing can be integrated at multiple stages, and doing so early is often recommended. Many teams incorporate automated checks into development environments, code review, and continuous integration pipelines so issues are surfaced before release. Running scans throughout the lifecycle, rather than only at the end, generally makes remediation less costly and helps prevent regressions.
What kinds of issues do automated tools tend to miss?
Automated tools generally struggle with issues requiring human judgment or context. Examples commonly cited include whether alternative text accurately conveys an image's meaning, whether headings and labels are descriptive, whether reading and focus order are logical, whether interactive components work correctly with assistive technology, and whether content is understandable. These typically require manual and assistive technology testing to evaluate.
Can automated testing help track accessibility progress over time?
Yes, within its limitations. Because automated checks are repeatable and can be run at scale, they are often used to monitor for regressions and to track trends in machine-detectable issues across releases. This can support an ongoing accessibility program, but the resulting metrics reflect only the subset of issues that tools can detect and should not be treated as a full measure of accessibility or conformance.

Common misconceptions

Passing an automated scan means a site is fully accessible or WCAG conformant.
Automated tools detect only a portion of accessibility issues. Many success criteria require human judgment, and passing automated checks does not guarantee an accessible experience or conformance. Manual review and assistive technology testing are also required.
Automated testing guarantees protection from legal claims.
Meeting automated checks does not ensure legal compliance or immunity from claims. Legal exposure is shaped by regulation, agency guidance, and case law, and organizations should consult qualified legal counsel rather than rely on tool output alone.
Any single automated tool catches all detectable problems.
Different tools implement different rule sets and may produce differing results, including false positives and false negatives. No single tool should be treated as a complete measure of even the machine-detectable portion of accessibility.

Best practices

Use automated testing as an early, repeatable first pass, but pair it with manual evaluation and testing with assistive technologies such as screen readers and keyboard navigation.
Integrate automated checks into development and continuous integration pipelines to catch regressions before release rather than only at the end of a project.
Treat automated results as candidate issues that require human verification, reviewing for false positives and investigating areas the tool cannot assess.
Map findings to specific WCAG success criteria and target level (AA is the level most commonly cited) to prioritize remediation consistently.
Supplement tool coverage with human review of context-dependent requirements, such as meaningful alternative text, logical reading order, and usable interaction flows.
Document testing methodology and remaining limitations, and consult qualified legal counsel and current agency guidance for compliance-related decisions rather than relying on scan results alone.