Introduction
Often, we’ll run into those accessibility bugs that aren’t so clear to understand. Sometimes you’ll run into screen reader issues, or maybe it’s a browser bug, or perhaps it’s an operating system issue.
This guide is meant to help anyone trying to debug an accessibility issue or discover why a problem is occurring.
Please note: I’ll be sure to keep this a living document and post updates as I go.
The a11y debugging guide
The steps below are some I use to help me debug an accessibility issue. I follow these steps when I think I’ve found a screen reader compatibility bug or a rare bug that has no documentation online.
Prerequisite 1 - You need to have a bug you're working on debugging
This document assumes you’ve found a bug already and are trying to debug it. So, this post requires some accessibility bugs filed into your bug tracking system.
For this post, if you need some inspiration, pretend you found a screen reader bug in the following environment:
- Screen Reader: VoiceOver
- Platform: IOS 15.5
- Browser: Safari.
Prerequisite 2- You already ran an automatic scan/tool.
This guide is for advanced accessibility issues. It assumes you have already tried the basics.
You should already have tried:
- Using an automatic checker such as “Axe dev tools.“
- You already validated the HTML such as through the Nu Validator.
Start of the Steps:
Step 1 – File the bug
File the bug. Ensure this bug is tracked and managed in your bug tracking system. Be sure to follow your organization’s standards for writing bugs. If you don’t have a standard bug writing template, here are some ideas below to get you started. I would recommend including details like:
- Screen Reader
- Platform
- Browser
- App Build
- Steps to Reproduce
- Any additional info
Step 2 – Check to see if this bug occurs anywhere else.
Goals: Determine if this issue is specific to your website or app.
- We want to check if this issue occurs anywhere else. So we might want to visit another website with a similar feature. Go online and test out an accessible version.
Requirements:
- You must use the same environment and combination that the reporter described in the bug. Everything is the same. The only thing that is changing is the website or environment. We’re trying to evaluate someone with a similar coding implementation on a different website.
Step 3 – Check if this occurs using a different assistive technology.
Goal: determine if this issue is related to your assistive tool.
- For example, let’s say this was a screen reader bug. We’re trying to determine if this issue is unique to that specific screen reader.
Requirements
- You must use the same environment but now with a different tool (e.g., screen reader). The only thing that changes is the tool; everything else must remain the same. For example, if you tested with NVDA, now do a test with JAWS or VoiceOver.
Step 4- Determine if it’s a browser/environment-related issue.
Goal: To determine if this is related to a specific browser or browser version.
- Is this only occurring in chrome? Or does it also happen in firefox or safari?
Requirements:
- You must change the browser but keep everything else the same. Use the same operating system and screen reader, but change the browser.
Step 5 – Check to see if this is reproducible by someone else.
Ask a friend or fellow a11y specialist to reproduce the bug. Be sure to give them instructions that are clear and easy to follow, and be sure to include details or credentials.
- Two people should be able to reproduce the bug, preferably three people.
If this is reproducible, move on to the next step.
Step 6 - Start disabling features and inspecting the code
First off, it’s critical to take a strategized approach here. We have to selectively disable things one by one to get an accurate baseline. If you disable multiple items, such as three things at once, how can you tell which one caused the issue?
This is a slow and steady process. Be patient. Listen to music or do some calming breathing exercises as you do this.
Step 7 - Disable CSS
Sometimes CSS affects what a screen reader says. Wow, crazy, right? A few examples of this are:
- In safari, if you apply the CSS of “list-style-type: none,” on a list item, the VoiceOver screen reader will not announce the element as a list. Scott o Hara also wrote about this in his post “Fixing Lists.”
- Developers sometimes use CSS to float things, leading to a reading order issue.
- I’m unsure if this is still the case, but I believe using “display: flex; on a data table, the screen reader won’t announce the element like a table. This does not allow a screen reader user to use the table semantics and disables the screen reader’s table navigation feature.
Once you disable CSS. Retest the page to confirm if the issue is resolved. If it’s not, CSS is not the issue.
Step 8 - Disable JS
Sometimes Javascript might cause issues. It’s hard to be a little prescriptive here as so many potential javascript scenarios can happen. Anytime you manipulate content dynamically there’s a risk of issues.
Once you disable Javascript, retest the page to confirm if the issue is resolved. If it’s not, the JS is not the issue.
Step 9 - Disable Images
Surprisingly I have run into issues where images have caused problems. Specifically, an image within an Iframe was causing a focus trap because it was lazy-loaded. The screen reader couldn’t understand the image due to the lazy load.
Once you disable images, retest the page to confirm if the issue is resolved. If it’s not, the images are not the issue.
Step 10 - Disable/Remove other things/Isolate even more...
Suppose disabling CSS, JS, and Images didn’t resolve your problem. The following steps are to continue to disable things as you see fit.
Remember to do it consciously, a scientific method approach of cause and effect. Channel your inner jack skellington.
- Start trimming down the DOM. For example, I might actually start deleting things one by one as I inspect the code to see if it’s causing issues.
- I would also copy the code and paste it into a separate environment. If I had a full page of 20 features, but my issue was on a specific table. I would extract the table and paste it into a codepen or onto a local HTML document and upload it to Github pages.
- Isolate the page however you see fit.
Step 11 - Look on the web
Please note: This step might go up earlier in the process for you. It really depends on your experience . I have this on the last stage as I usually don’t have to look things up for most issues as I’ve experienced lots of issues. With that out of the way, let’s continue.
Time do some research online. We’ve exhausted our personal efforts.
- Do a quick google search. It’s critical to search with specific details and keywords.
- Look through the documentation or existing support pages. For example, if you go onto NVDA’s GitHub page, you can see existing issues for NVDA.
- Search through common a11y slack channels like the web-a11y slack channel. I would also ask a question here.
Step 12 - File an external dependency bug
Please file a bug stating that it’s an external dependency bug. So let’s say you found an NVDA bug. You should go to the NVDA GitHub page (opens window) and say, “Hey, what the heck, there’s a bug.”
I would flag a bug there and see what they say. Maybe it’s a known issue, perhaps it’s a new bug, and you contributed to the community.
Here’s a table of all the potential places you might want to file an accessibility external dependency bug.
This table might require horizontal scrolling if you’re on mobile
| Bug Tracker | URL | Bug Type |
|---|---|---|
| JAWS | https://github.com/FreedomScientific/VFO-standards-support/issues | Screen Reader |
| NVDA Issues | https://github.com/nvaccess/nvda | Screen Reader |
| Apple Bug Reporter | https://feedbackassistant.apple.com/ | Screen Reader Assistive Technology Native iOS/OS X |
| Firefox | https://bugzilla.mozilla.org/ | Browser |
| Webkit Bugzilla including Safari | https://bugs.webkit.org/ | Browser |
| Android | https://source.android.com/source/report-bugs.html | OS |
| Open Radar | http://www.openradar.me | Screen Reader Assistive Technology Native iOS/OS X |
| Chromium Bug Tracker | https://code.google.com/p/chromium/issues/list | Browser |
| Internet Explorer Feedback | https://connect.microsoft.com/ie/ (Microsoft Connect has been retired) | Browser |
| TalkBack | https://www.google.com/accessibility/get-in-touch.html | Screen Reader |
| VoiceOver | Apple welcomes comments and inquiries about the accessibility of their products. Please accessibility@apple.com. This is the email to use/recommended for best results regarding bugs and issues with VO specifically. | Screen Reader |
| GNOME (Orca) Bugzilla | https://bugzilla.gnome.org/enter_bug.cgi?product=orca | Screen Reader |
| Dragon Naturally Speaking | Tech Support | Assistive Technology |
| ZoomText | Ai Squared Support | Assistive Technology |
| WAI-ARIA Specs | W3C ARIA GitHub Repository | Specs |
| Microsoft Edge | Microsoft Edge Issue Tracker | Browser |
Conclusion
Debugging a bug can be tricky. Whether you’re a developer or an accessibility tester, having a guide to debugging problematic accessibility issues should help.
Suppose you notice anything in the blog post that can be improved. Please leave a comment or reach out to me via LinkedIn. I’m always open to feedback.
I hope this helps!
Please note: I plan to upload a video here in the coming weeks. My goal is to provide a video for all posts. So keep a lookout for that soon
Thanks! – Gio
Memes for fun
Casual programmer memes.
Updates
- 1/2/2023 Update: I added a prerequisite #2 which notes users should have already used an automatic scanner and HTML validator. Shout out to my colleague Kim Collins for recommending this.
9 responses to “The Ultimate Guide to Debugging Accessibility Issues”
Great article! Awesome job explaining step by step testing! Well done!!
I’m curious if you’ve used the accessibility tree for debugging. Do you have any suggestions around that?
Hi Caryl! Thanks for the comment. Happy to discuss this with you on Linkedin. I did write a blog post in the past on how to use the Accessibility Tree
I would add steps to also look at upstream sources of the bug. Is the bug just something in your site, or does it occur in a library that you use. In complex modern sites there are lots of places that bugs can occur.
I’d add a caveat that it may not even be a bug, but just the way a particular screen reader works and users would totally be okay with it. The only way to figure that out is to talk to actual screen reader users. It can save you from going down a rabbit hole of debugging and fixing what isn’t even a real problem for users.
threexvideo.com
vidozahost.com
short4cams.com
These are actually great ideas in regarding blogging. You have touched some nice points here.
Any way keep up wrinting.