Spring is here and we have Global Accessibility Awareness Day (GAAD) on our minds! WebAIM recently conducted a survey to uncover the most common accessibility failures on website homepages. The results are featured on the GAAD homepage. It got us thinking about how our Digital Accessibility Coordinators (DACs) across Minnesota state agencies conduct initial testing for website accessibility failures, the tools they use, and how they decide to fix the issue or determine to leave it alone. 

State of Minnesota Standard for Website Accessibility Testing

It is important to understand the specific elements that must be present in a web page to ensure an accessible experience. The State of Minnesota has a Digital Accessibility Standard, which requires our digital content, including state web pages, to meet WCAG 2.0, Level AA and Section 508 of federal law.

If you want to dig deeper into understanding how these standards were developed (and the State’s planned update to WCAG 2.1 on July 1, 2024), view our February 2024 blog “Update to State of Minnesota Digital Accessibility Standard Launches July 1.”

Accessibility testing should happen early and often during all System Development Life Cycle (SDLC) processes: requirements analysis, design, implementation, testing and deployment. Testing for accessibility along the way can help to identify problems early in the process and ensure every state agency is delivering accessible government digital content for everyone.

Meet three of our DACs and this month’s contributors

Let’s meet the DACs that provided their expertise and recommendations for the tools they use to perform web accessibility testing. Between the three, they have more than 60 years of website accessibility testing experience!

• Emily Lu, DAC, Minnesota Department of Transportation.
• JoAnn Rautio, CSM, DHS Certified Trusted Tester v5, Quality Assurance and DAC, MNIT partnering with the Minnesota Department of Human Services.
• Kim Wee, CPWA, DHS Certified Trusted Tester v5, Webmaster and DAC, MNIT partnering with the Department of Education. 

Are you a State of Minnesota employee and want to learn who your agency’s DAC is? Reach out to us by email at mnitaccessibility@state.mn.us and we would be happy to connect you.

DAC Recommendations for Website Accessibility Testing Tools

These web accessibility evaluation tools are software programs or online services that help determine if web content meets accessibility standards. Each DAC shared: 

• how they discovered their testing tools, 
• if training was required to begin to use it, 
• how the tool works, 
• their favorite and most challenging features of the tool(s), and 
• how the tool outputs accessibility failure results.

Tool Background

*We assume you already own a computer with a keyboard, which makes your keyboard a tool that is available to use for testing with no additional cost.

Tool Functionality

Each DAC provided details on how the tool performs, with their favorite features and biggest challenges, if any.

Keyboard 

Favorite features:

• Ease of use.
• Free.
• Identifies components on the page that need further investigation. 
  • Can the user get to all links, buttons, menu items, and other interactive elements?
  • Can the user see where they are (“Visible focus” on content)?
  • Does the focus order make sense?
  • Do all hoverable components (like a pop-up on an info icon) also receive keyboard focus?

Browser DevTool – Accessibility Tree

Favorite features:

• Ease of use.
• Free.
• Reviews the code for Accessible Rich Internet Applications (ARIA).
• A reliable way to troubleshoot issues identified during the testing process.

Challenges:

• Advanced technical knowledge may be needed to understand what the Accessibility Tree identifies as issues and how those relate to what is being tested.

Accessibility Insights for Web Browser Extension

Favorite features:

• Can find common accessibility issues very quickly.
• Verifies web pages are compliant up to WCAG 2.1, Level AA.

Challenges:

• This is a newer testing tool, and DACs are still learning how to use the tool to its full advantage.

WAVE Browser Extension

Favorite features:

• Web-based with no application download required. There’s also a small browser extension provided at no cost.
• The report is designed for easy comprehension, offering explanations and references to WCAG guidelines and why it matters.
• Provides a color Contrast tab, allowing testers to easily select the appropriate colors to achieve the required contrast ratio.
• Provides an Order tab with all navigable page elements listed. 
• Provides a Reference tab that verifies for WCAG.

Challenges:

• Like other automated scanning tools, WAVE may overlook certain issues or generate false positive results. 
• Navigating the overlay of icons can be challenging, particularly when inspecting invisible elements on the page.

aXe DevTool Browser Extension

Favorite features:

• Ease of use.
• Free.
• Can be used by developers in their local environment – all you need is a browser.
• Provides results for ARIA issues.

Challenges:

• Takes time to learn and interpret the results. 
• Advanced technical knowledge may be needed to understand what the aXe DevTool identifies as issues.

JAWS Inspect

Favorite features:

• Has a speech viewer which provides a live log of JAWS speech. 
• Full page reports which identify issues at the component and element level.
• Allows testers to check for duplicate button labels (like JAWS keyboard commands).

Challenges:

• Takes time to learn and interpret the results.

Tool Output

Keyboard

The output comes from the tester’s interaction (this is a manual process). There is no screen with output results. You will want to have a way to capture your results (e.g., in an Excel spreadsheet).

Browser DevTool – Accessibility Tree

The DevTool opens within any browser. To launch DevTool in a browser window, either select F12 or right click and select “Inspect.”  Within the Accessibility tab is the Accessibility Tree. It contains tabs for ARIA Attributes, Computed Properties and Secure Order Viewer. 

Note: The Accessibility tab does not allow you to make edits to the page nor export its content. You will need to keep a report of your findings to reference when you make edits or submit a request for edits.

WAVE Browser Extension

WAVE will display the original web page with embedded icons and indicators that highlight the accessibility components. The output results are divided into six categories: Summary, Details, Reference, Order, Structure and Contrast. 

To review the details about each icon added to the scanned web page, view the Details tab. To pinpoint the specific issue or feature of an icon, click or tab+enter on any icon and review the details provided under the Reference tab.

Note: WAVE does not allow you to make edits to the page nor export its content. You will need to keep a report of your findings to reference when you make edits or submit a request for edits.

Accessibility Insights for Web Browser Extension

This tool offers a launch pad with different types of tests to perform: FastPass, Quick Assess, Assessment and Ad Hoc Tools. Each option provides a different output that allows you to navigate through identified accessibility issues.

aXe DevTool Browser Extension

aXe opens in your browser. You can select the "Scan ALL of my page” or “Scan PART of my page” button and then review the list of identified accessibility issues.

JAWS Inspect

JAWS provides output in a few ways: screenshot, code snippet and a unique tracking ID. Note: You do not need to have the JAWS software to perform JAWS Inspect.

Addressing Accessibility Testing Failures

Emily Lu

For common errors with straightforward solutions, like low contrast, missing alt text, empty links, unclear link text, empty buttons, and missing labels, I promptly request that web editors fix them.

For more complex errors [or issues which] require more time to fix, I usually recommend conducting a risk assessment and prioritize based on the impact and severity of the issues. If this issue occurs frequently and critically impacts the usability of our service, then we will want to address these issues first, before moving on to issues that have a lesser impact on accessibility and usability.

JoAnn Rautio

Our projects are broken out into sprints. During a sprint, the issue is documented as a SPRINT BUG and fixed within that sprint. When found during regression testing it is documented as a work item called BUG and assigned to a future sprint based on risk level.

There are times we cannot fix an issue due to third-party tools or widgets that are used in an application. We create a work item called Analysis. We research, discuss, and document whether it can be fixed or needs a vendor-updated release or replacement.

Kim Wee

We prioritize into high impact and low effort repairs:    

• High impact repairs (user impact, multiple pages, navigation, home page, etc.).
• Low effort repairs (less time to fix, requires less testing).

Final Takeaways

There isn’t one tool that does it all, so having a library of tools can be very beneficial to ensure a well-rounded website accessibility testing process. While tools are excellent to help aid the testing process, having a knowledgeable person perform manual testing is also a necessity to ensure your testing process is sufficient to capture accessibility failures for your web pages.

It’s also important to understand the impact  of the identified accessibility issues on your web pages. Have a process on how to address each one, whether to let them be (and the reason/rationale behind that decision) or to fix them  and how/when that occurs. The impact is on a spectrum from low to high and will affect different groups. For example each of these issues have a different potential impact on users:

• A screen reader or keyboard user cannot complete a form because the “Submit” button isn’t properly identified and not able to be tabbed to.
• Color contrast failure may require a website redesign that involves several agency departments to review and approve.
• Missed links or alt text needs a content manager to fix.

We hope this article provides insight on how to spruce up your own website accessibility testing tools and checklist to ensure accessible government digital content for everyone.

Additional Resources 

• W3C’s Evaluating Web Accessibility Overview web page.
• WCAG 2.0 Quick Reference Guide.
• WCAG 2.1 Quick Reference Guide.
• Tools:
  • Accessibility Insights
  • WAVE
  • aXe
  • JAWS Inspect for Chrome
• State of MN employees: Office of Accessibility extranet: Tools page