denkwerk TechTalk

A11y-AI: AI for empathic assessment of accessibility

"Accessibility is still not a matter of course in the realization of software projects.
A11y-AI uses the possibilities of Large Language Models to present the effects of barriers in the web on people with disabilities in a sensitive and personal way.
In this way, we want to create empathy for the realization of accessible web projects."

Miguel Franken Senior Software Developer at denkwerk

But how does it work? The system consists of three pillars: a persona, the accessibility check and the context, i.e. the content of the website. All of this is brought together using GPT to provide an analysis in natural language, which the persona appears to have written itself.

The solution: an artificial persona

In a first step, we have therefore developed an artificial person ("persona") that has a background story and a certain type of restriction. Claudia was developed as the first persona – others will follow: Claudia is 63, an architect, and suffers from macular degeneration, a disease that severely impairs central visual acuity: People with this disease can only see sharply at the edge of their field of vision, and the disease leads to blindness in the medium term. The persona is therefore severely visually impaired.

In a second step, the system uses Axe-Core tool to analyze the website entered and outputs a report. Axe-Core currently analyzes 103 accessibility rules, but the generated A11y report is very technical. Therefore, our development team has designed a preprocessing that translates the technical output into non-technical language.

Analysis of the context ensures appealing comments

In a third step, GPT analyzes the content of the website and provides the context. In the presentation, for example, an online music store was presented: Claudia should contextually refer to the website in her response, which is why this step makes sense. This not only makes the system appear more authentic, but also allows it to deal with the barrieres that arise in a humorous way.

Together, A11y-AI creates an accessibility report in natural language, which appears to have been created by "Claudia": she points out problems and explains why they are relevant for her and possibly for other people with visual impairments. The system also recognizes rule violations that are so serious that assistance systems cannot be used, for example. The text is divided into three sections: Introduction, the actual report – with details of the barriers found – and the conclusion.

And so, after entering a website URL, "Claudia" determines the difficulties that can arise for a person with a severe visual impairment. Through the persona "Claudia", the system is empathetic and shows that the web must be made accessible. This approach helps decision-makers better understand why the optimization makes sense – and thus people with disabilities have a much better online experience.