Why your AI accessibility tools still miss critical issues and how to build a smarter human-in-the-loop strategy.
Digital teams are increasingly leaning on AI-powered accessibility tools like Deque axe and mabl to keep up with release velocity and compliance pressure. These tools are fantastic at catching a wide range of low-hanging WCAG violations in seconds. But if you have ever run a full manual audit after an “all green” automated report, you already know the truth:
AI accessibility tools are necessary, but nowhere near sufficient.
In this post, I will unpack why tools like axe and mabl systematically miss critical issues that experts still catch manually, and how to combine automation, AI, and human judgment into a practical, scalable accessibility strategy.
The Promise (and Reality) of Automated Accessibility
Tools like Deque axe and unified testing platforms such as mabl are built on powerful rule engines and, increasingly, AI-assisted capabilities. They scan your DOM, attributes, ARIA roles, and styles, then map those signals to WCAG success criteria.
In practice, these tools excel at:
- Finding missing labels, alt text, and ARIA attributes.
- Flagging obvious color-contrast problems.
- Catching broken form associations and non-focusable interactive elements.
- Preventing regressions by running on every PR, build, or nightly run.
For busy teams, that level of coverage is a game changer. But it also creates a dangerous illusion: passing automated tests does not mean a product is accessible. It simply means the problems that are easy for machines to detect look good right now.
Why AI Accessibility Tools Still Miss Critical Issues
1. They are bound to rules, not judgment
Automated engines such as axe-core apply large WCAG rulesets to the current DOM and code. They are deliberately conservative to avoid false positives. Anything that requires human interpretation is either skipped or heavily simplified.
Consider questions like:
- Is this link text meaningful in context?
- Is this error message actually helpful?
- Does this visual grouping make sense when you hear it instead of see it?
Those are judgment calls. Today’s AI and rule engines do not understand intent, mental models, or the broader task a user is trying to accomplish. So, they stay safely in the world of syntax, attributes, and simple patterns.
2. Information structure and semantics are hard to infer
Many high-impact accessibility failures are about information and relationships, not just individual elements:
- Headings that look right but are not coded as headings.
- Missing or misused landmarks (main, nav, complementary, etc.).
- Custom widgets that expose the wrong role, name, or state to assistive technologies.
Axe or mabl may see ARIA sprinkled around and assume things are reasonable. A screen reader user, on the other hand, will quickly discover that the page outline makes no sense, landmarks are missing, or a fancy custom control never announces its state.
These problems only show up when someone navigates like a real user via keyboard and a screen reader and listens to the experience instead of looking at it.
3. Usability, cognition, and content quality are out of scope
Accessibility is not just about whether a user can technically operate the UI. It is also about whether they understand what is happening without unnecessary cognitive load.
AI tools do not assess:
- Whether instructions are concise, clear, and free of jargon.
- Whether error messages tell you exactly what to fix or just say “invalid input”.
- Whether the visual and reading order match a logical story.
- Whether content density overwhelms users with cognitive or learning disabilities.
Manual experts routinely flag these issues, because they are experiencing the product as humans, not as parsers. Compliance engines simply are not designed to make these kinds of calls.
4. Dynamic behavior and multi-step flows are fragile
Even when integrated into end-to-end tests, automated checks usually examine one DOM state at a time. Many of the nastiest accessibility bugs only appear across steps:
- A modal that initially traps focus correctly, then loses it when a validation error appears.
- A wizard that silently resets the user’s position after a failed step.
- Toast messages or status banners that appear visually but are never announced.
Unless your tests happen to navigate through those exact sequences and pause long enough in the right state for checks to run, the tool will not see the problem.
5. Custom design systems and edge cases resist generic rules
Modern design systems are full of custom widgets: combo boxes, virtualized tables, drag-and-drop lists, complex charts, and canvas-based visualizations. Their accessibility depends on bespoke keyboard handling and ARIA contracts.
Rule engines can only infer so much from markup. To avoid noisy false positives, they are intentionally conservative, which means they often under-report real issues in bespoke components.
This is where expert reviews and testing with real assistive-technology users are non-negotiable.
6. Workflow and coverage limits in CI/CD
Platforms like mabl shine when you piggyback accessibility checks on your existing browser tests and pipelines. That is a huge win, but it also means anything not covered by your functional suite is invisible to accessibility checks.
Edge-case paths, personalized states, and error flows often have no automated coverage. Expert auditors, in contrast, deliberately stress those dark corners. They explore the what-if paths your happy-path tests never touch.
What axe and mabl Are Actually Great At
Despite all these limitations, tools like axe and mabl are still essential to a modern accessibility practice, especially as AI-assisted testing matures.
Fast, repeatable baseline
Use them to:
- Scan every page or key user journey on each PR or deployment.
- Enforce non-negotiables like alt text, labels, and contrast.
- Catch and block regressions on issues you have already fixed.
This is where machines beat humans every day: volume, speed, and consistency.
AI-assisted test creation and maintenance
Mabl adds another valuable layer by using AI to:
- Generate and update browser tests as the UI evolves.
- Self-heal selectors and locators when markup changes.
When you wire accessibility checks into those tests, you get AI-maintained coverage of your most important flows with almost no ongoing scripting effort.
Semi-automated checks
Deque’s guided testing concepts sit in a useful middle ground. The tool asks targeted questions, you perform specific actions, and it records results in a structured way. It is not fully automated, but it:
- Encodes expert checklists into repeatable flows.
- Makes more nuanced checks accessible to non-specialists.
Think of this as AI supporting human reviewers, not replacing them.
A Practical Human-in-the-Loop Strategy
So how do you combine all of this into something that works sprint after sprint?
1. Make automation your accessibility guardrail
Start by using axe and mabl to enforce machine-detectable basics everywhere:
- Run scans locally and in pre-commit hooks.
- Fail PRs when critical violations are introduced.
- Run a broader nightly or pre-release suite for your top user journeys.
This keeps the baseline clean and prevents the same obvious issues from coming back again and again.
2. Focus manual effort on real user tasks
Reserve expert time for realistic, end-to-end scenarios:
- Completing checkout or registration.
- Configuring a complex report or dashboard.
- Filling and submitting multi-step forms.
Have testers:
- Use only the keyboard and screen reader.
- Pay close attention to focus, announcements, and error handling.
- Judge whether content and interactions are actually understandable.
This is where your biggest, most user-visible wins will come from.
3. Close the loop between humans and tools
Every time a manual audit finds an issue, ask two questions:
- Can we fix this at the design-system level?
- Can we teach our tools to catch this next time?
If yes, patch the component and add automated checks around it. Add custom rules, linters, or patterns to axe, mabl, or your CI checks. Over time, your AI and rule engines become smarter, and expert time shifts from re-finding the same class of bugs to discovering new ones.
4. Use dashboards to prioritize, not just to report
Both Deque-based setups and mabl provide dashboards showing which pages, flows, and components accumulate the most issues. Use these views to identify hotspots in your product, prioritize fixes by severity and user impact, and demonstrate progress to stakeholders with hard numbers.
5. Test with users who rely on assistive technologies
Finally, and most importantly, bring in people who use screen readers, magnifiers, switch devices, and keyboard-only navigation every day. Ask them to perform the same core tasks you care about.
They will surface problems no tool or expert anticipated: unexpected reading orders, confusing mental models, and places where technically correct behavior is still unusable in practice.
The Bottom Line
AI and rule-based accessibility tools like Deque axe and mabl are foundational for any serious accessibility program. They give you speed, scale, and regression protection, and they are only getting better.
But they are not a substitute for human judgment.
Treat them as powerful guardrails and accelerators. Then invest intentionally in expert audits, task-based manual testing, and sessions with users who rely on assistive technologies. That is how you transform an all-green automated report into a product that is genuinely usable and inclusive for everyone.
Authored by – Vincent Emerald
