devices over wood blog 2.jpg

    Do Free ADA Web Testing Tools Help or Hurt Accessibility?

    by Jason Taylor

    With the surge of ADA Web lawsuits in 2018 and early 2019, companies are seeking answers and new tools to make their websites and apps ADA compliant. Automated accessibility testing tools are available in large supply and are often the first stop for developers to understand the accessibility of their website/s.

    But do these tools give a true assessment? Search “how accessible is my website?” online and you can easily find a number of free automated testing tools like WAVE or extensions. At UsableNet, we provide a free version of our enterprise testing platform UsableNet AQA.   With these tools, you can test a single page at a time to see how many accessibility issues are found. You might do a quick calculation across all the pages you have and conclude, “oh we are not too bad” or “Wow, we have some work to do.”

    But these results can be misleading in terms of true accessibility - the ability for all users to access, navigate and understand how to complete tasks on your Website.

    The latest most popular tool like this is Chrome’s built-in Lighthouse audit tool which tests for four broad areas of website quality, one of them being accessibility. The difference between Chrome's tool and others is that it goes so far as to give the user a “percentage” of Accessibility for their website, and it carries the credibility of Google.

    But a number of Accessibility experts have criticized this scoring method. Since it's release, developers have written articles showing how clearly inaccessible websites can still score 100% with the Chrome Lighthouse rating system.

    How can a developer get 100% accessibility score for a web site that is in-accessible? 

    Automated testing tools only test for 20-30 % of the 38 Success Criteria that the WCAG 2.0 AA (the prevailing standard) requires to ensure Accessibility. This means that when you pass the automatic test with ANY of these testing solution you are actually starting at 30% and still have another 28 or so Success Criteria to check. An example of one of these checks not covered by auto testing is- check for keyboard traps -and if your web site fails that in key areas of any user flows, no screen readers (and other assistive tech) can complete user tasks - and you have 27 more SC to go. You get the idea.

    Why are these tools so popular? And are they good for the overall effort of accessible design and UX?

    Let’s start with a Quick History of Automated Accessibility Testing tools and why they were developed.

    In the Beginning, Around 1998 there was “Bobby” The first free service much like WAVE (over the following years it was purchased by watchfire, which was then purchased by IBM and retired from commercial availability by IBM).

    In 2000, UsableNet released our own accessibility and Usability testing product. This was quickly followed by WAVE, SSB (now Level Access) and Deque in 2001. By 2004 there were six main Accessibility Testing Tools, read more.

    Early testing was driven by Section 508 (Federal Accessibility Law) and to help individual developers, of mainly government sites and educational sites, effected by Section 508. These sites were mostly a collection of static html pages organized around simple navigation.

    In this early context, automated testing tools added significant value. HTML was the primary code generating the final DOM of a page and the pages were being developed one page at a time by a developer using Macromedia Dreamweaver, Microsoft FrontPage or Adobe Pagemaker (Ah, the memories). Users started at one page and navigated to another page.

    Today CSS (styles) and Scripting (interactions) play a much larger role in the final website, particularly with Retail, Travel, Banking and Healthcare sites that offer functionality not just content (we used to call these Web Apps). These cannot be tested by automated testing in the same way as simple web content sites. More importantly development teams don’t work the same way. Dev teams work on user flows and widgets to provide user features, not flat pages of content that are connected by hyperlinks. In addition, these functional websites are now generated using engines that also have customization applied to them at run-time. This means that the CSS, Javascript and even element ID's could be different in the final page compared to originally written.

    As the development options have increased over the years so too has the complexity of the accessibility review and remediation.

    So are automated testing tools good to use?

    The quick answer is yes. But within the context of an overall process and strategy to design, develop, test and maintain accessibility.

    To be clear, 90% of all websites in the world are content, informational or CMS based, so automated accessibility testing is a very useful tool for these sites as the majority of issues, in volume, will be discovered and the fix is typically a one person or one department activity.

    More functional based sites involve more people (UX, development teams, project management, marketing, QA, compliance Etc), and the value of an accessibility testing platform is based much more around reducing time throughout the overall process across multiple people and teams.

    The accessibility testing platforms required by this functional sites need to integrate with current systems (Ticket systems such as JIRA, Release Management such as Jenkins, etc) and remove effort and time in discovering functional accessibility issues, remediation discussions, remediation verification and on-going maintenance.

    The companies that have been providing accessibility testing for the longest including UsableNet, Level Access and Deque Systems all still maintain a simple free one-page-testing tool for the accessibility and web community to use,  but they have developed bigger and better full systems that extend accessibility into DevOps and across modern enterprise.

    These systems reflect the need for accessibility to become a team activity from design, through prototype build, development, deployment and maintenance.

    For UsableNet’s part, our UsableNet AQA Testing platform brings to larger teams the following 5 areas that are required today beyond just simple testing a page. This helps organizations integrate accessibility throughout the development life-cycle:

    1. AQA platform captures and stores the tested DOM of all the user journey and interactions, identifies the exact portion of code that needs changing, provides guided manual review and ability to add notes and comments. This means no confusion on where an issue is and no need to spend time replicating issues.
    2. AQA has Enterprise grade API that allows it to send and receive information from the tools you use. JIRA Ticketing, CI, test management etc. This allows teams to work with tools they know and makes it easier to integrate into your existing processes.
    3. AQA tests single pages, scans entire sites and also can test user journeys such as where a user navigates and uses a site. AQA interacts with forms, logins and the complex functionality you find across most websites today. Testing is much easier to manage and you can find issues that other tools might miss.
    4. AQA finds all accessibility issues, not just the easy automated ones. AQA will call out all the relevant guidelines that apply to a site and through a built-in screen-reader, guide the user to determine a pass, fail or solution. This means you are covering all bases. 
    5. AQA provides a team a range of documentation, dashboards and reports that fit the needs of larger teams. Individuals can track improvements over time. Team leads can track activity, and Compliance teams can generate reports to satisfy legal requirements.

    The majority of these features are available in our free online Version of UsableNet AQA for you to try one page at a time, but the real power comes when an enterprise integrates this solution across its teams, systems and methodology.

    If you haven't taken a look at UsableNet AQA lately, now's the time. Get in touch to learn more or request your free consultation. 

    Is your website WCAG 2.0 or 2.1 compliant? Test your website with AQA.


    ADA Website Compliance

    Need to improve digital usability, accessibility or performance? We can help.
    Partner with us. Get in touch.