Introduction

Often, we’ll run into those accessibility bugs that aren’t so clear to understand. Sometimes you’ll run into screen reader issues, or maybe it’s a browser bug, or perhaps it’s an operating system issue.

This guide is meant to help anyone trying to debug an accessibility issue or discover why a problem is occurring.

Please note: I’ll be sure to keep this a living document and post updates as I go.

 

The a11y debugging guide

The steps below are some I use to help me debug an accessibility issue. I follow these steps when I think I’ve found a screen reader compatibility bug or a rare bug that has no documentation online.

Prerequisite - You need to have a bug you're working on debugging

This document assumes you’ve found a bug already and are trying to debug it. So, this post requires some accessibility bugs filed into your bug tracking system.

For this post, if you need some inspiration, pretend you found a screen reader bug in the following environment:

  • Screen Reader: VoiceOver
  • Platform: IOS 15.5
  • Browser: Safari.

Step 1 – File the bug

File the bug. Ensure this bug is tracked and managed in your bug tracking system. Be sure to follow your organization’s standards for writing bugs. If you don’t have a standard bug writing template, here are some ideas below to get you started. I would recommend including details like:

  • Screen Reader
  • Platform
  • Browser
  • App Build
  • Steps to Reproduce
  • Any additional info

Step 2 – Check to see if this bug occurs anywhere else.

Goals: Determine if this issue is specific to your website or app.

  • We want to check if this issue occurs anywhere else. So we might want to visit another website with a similar feature. Go online and test out an accessible version. 

Requirements:

  • You must use the same environment and combination that the reporter described in the bug. Everything is the same. The only thing that is changing is the website or environment. We’re trying to evaluate someone with a similar coding implementation on a different website.

Step 3 – Check if this occurs using a different assistive technology.

Goal: determine if this issue is related to your assistive tool.

  • For example, let’s say this was a screen reader bug. We’re trying to determine if this issue is unique to that specific screen reader.

Requirements

  • You must use the same environment but now with a different tool (e.g., screen reader). The only thing that changes is the tool; everything else must remain the same. For example, if you tested with NVDA, now do a test with JAWS or VoiceOver.

Step 4- Determine if it’s a browser/environment-related issue.

Goal: To determine if this is related to a specific browser or browser version.

  • Is this only occurring in chrome? Or does it also happen in firefox or safari?

Requirements:

  • You must change the browser but keep everything else the same. Use the same operating system and screen reader, but change the browser.

Step 5 – Check to see if this is reproducible by someone else.

Ask a friend or fellow a11y specialist to reproduce the bug. Be sure to give them instructions that are clear and easy to follow, and be sure to include details or credentials.

  • Two people should be able to reproduce the bug, preferably three people.

If this is reproducible, move on to the next step.

Step 6 - Start disabling features and inspecting the code

First off, it’s critical to take a strategized approach here. We have to selectively disable things one by one to get an accurate baseline. If you disable multiple items, such as three things at once, how can you tell which one caused the issue?

This is a slow and steady process. Be patient. Listen to music or do some calming breathing exercises as you do this.

Step 7 - Disable CSS

Sometimes CSS affects what a screen reader says. Wow, crazy, right? A few examples of this are:

  • In safari, if you apply the CSS of “list-style-type: none,” on a list item, the VoiceOver screen reader will not announce the element as a list. Scott o Hara also wrote about this in his post “Fixing Lists.”
  • Developers sometimes use CSS to float things, leading to a reading order issue.
  • I’m unsure if this is still the case, but I believe using “display: flex; on a data table, the screen reader won’t announce the element like a table. This does not allow a screen reader user to use the table semantics and disables the screen reader’s table navigation feature.

Once you disable CSS. Retest the page to confirm if the issue is resolved. If it’s not, CSS is not the issue.

Step 8 - Disable JS

Sometimes Javascript might cause issues. It’s hard to be a little prescriptive here as so many potential javascript scenarios can happen. Anytime you manipulate content dynamically there’s a risk of issues.

Once you disable Javascript, retest the page to confirm if the issue is resolved. If it’s not, the JS is not the issue.

Step 9 - Disable Images

Surprisingly I have run into issues where images have caused problems. Specifically, an image within an Iframe was causing a focus trap because it was lazy-loaded. The screen reader couldn’t understand the image due to the lazy load.

Once you disable images, retest the page to confirm if the issue is resolved. If it’s not, the images are not the issue.

Step 10 - Disable/Remove other things/Isolate even more...

Suppose disabling CSS, JS, and Images didn’t resolve your problem. The following steps are to continue to disable things as you see fit.

Remember to do it consciously, a scientific method approach of cause and effect. Channel your inner jack skellington.

  • Start trimming down the DOM. For example, I might actually start deleting things one by one as I inspect the code to see if it’s causing issues. 
  • I would also copy the code and paste it into a separate environment. If I had a full page of 20 features, but my issue was on a specific table. I would extract the table and paste it into a codepen or onto a local HTML document and upload it to Github pages.
  • Isolate the page however you see fit.

Step 11 - Look on the web

Please note: This step might go up earlier in the process for you. It really depends on your experience . I have this on the last stage as I usually don’t have to look things up for most issues as I’ve experienced lots of issues. With that out of the way, let’s continue.

Time do some research online. We’ve exhausted our personal efforts.

  • Do a quick google search. It’s critical to search with specific details and keywords.
  • Look through the documentation or existing support pages. For example, if you go onto NVDA’s GitHub page, you can see existing issues for NVDA.
  • Search through common a11y slack channels like the web-a11y slack channel. I would also ask a question here.

Step 12 - File an external dependency bug

Please file a bug stating that it’s an external dependency bug. So let’s say you found an NVDA bug. You should go to the NVDA GitHub page (opens window) and say, “Hey, what the heck, there’s a bug.”

I would flag a bug there and see what they say. Maybe it’s a known issue, perhaps it’s a new bug, and you contributed to the community.

Here’s a table of all the potential places you might want to file an accessibility external dependency bug.

This table might require horizontal scrolling if you’re on mobile

Bug Tracker URL Bug Type
JAWS https://github.com/FreedomScientific/VFO-standards-support/issues

Screen Reader
NVDA Issues https://github.com/nvaccess/nvda Screen Reader
Apple Bug Reporter https://feedbackassistant.apple.com/ Screen Reader Assistive Technology Native iOS/OS X
Firefox https://bugzilla.mozilla.org/ Browser
Webkit Bugzilla including Safari https://bugs.webkit.org/ Browser
Android https://source.android.com/source/report-bugs.html OS
Open Radar http://www.openradar.me Screen Reader Assistive Technology Native iOS/OS X
Chromium Bug Tracker https://code.google.com/p/chromium/issues/list Browser
Internet Explorer Feedback https://connect.microsoft.com/ie/ (Microsoft Connect has been retired) Browser
TalkBack https://www.google.com/accessibility/get-in-touch.html Screen Reader
VoiceOver Apple welcomes comments and inquiries about the accessibility of their products. Please accessibility@apple.com. This is the email to use/recommended for best results regarding bugs and issues with VO specifically. Screen Reader
GNOME (Orca) Bugzilla https://bugzilla.gnome.org/enter_bug.cgi?product=orca Screen Reader
Dragon Naturally Speaking Tech Support Assistive Technology
ZoomText Ai Squared Support Assistive Technology
WAI-ARIA Specs W3C ARIA GitHub Repository Specs
Microsoft Edge Microsoft Edge Issue Tracker Browser

Conclusion

Debugging a bug can be tricky. Whether you’re a developer or an accessibility tester, having a guide to debugging problematic accessibility issues should help.

Suppose you notice anything in the blog post that can be improved. Please leave a comment or reach out to me via LinkedIn. I’m always open to feedback.

I hope this helps!

Please note: I plan to upload a video here in the coming weeks. My goal is to provide a video for all posts. So keep a lookout for that soon

Thanks! – Gio

Memes for fun

Casual programmer memes.

Man on phone named - Debugging. Text saying: I don't know where you are. I don't know how you work, but I will find you, and I will fix you.
2 panel Meme: On top a person pensive looking at computer saying "It doesn't work...why?" and underneath the same image but it says "it works.... why?"

Comments (Markdown Supported)

5 responses to “The Ultimate Guide to Debugging Accessibility Issues”

  1. Anna Smith says:

    Great article! Awesome job explaining step by step testing! Well done!!

  2. Caryl says:

    I’m curious if you’ve used the accessibility tree for debugging. Do you have any suggestions around that?

  3. Mike Gifford says:

    I would add steps to also look at upstream sources of the bug. Is the bug just something in your site, or does it occur in a library that you use. In complex modern sites there are lots of places that bugs can occur.

  4. Kate Kalcevich says:

    I’d add a caveat that it may not even be a bug, but just the way a particular screen reader works and users would totally be okay with it. The only way to figure that out is to talk to actual screen reader users. It can save you from going down a rabbit hole of debugging and fixing what isn’t even a real problem for users.

Leave a Reply

Your email address will not be published.

Blog Categories

Search for a post

Blog Categories