Enhancing Web Accessibility Remediation with LLM-based IDE Extensions

Abstract

AI coding assistants offer opportunities for web accessibility support, but evidence on how accessibility-focused guidance affects code accessibility and developer experience remains limited. This thesis presents A11yHelper, a Visual Studio Code extension integrated with GitHub Copilot Chat, and compares its accessibility quality, perceived usability, and tool preference with GitHub Copilot and CodeA11y. The evaluation combined a fixed-prompt experiment and a controlled user study with 18 participants. Both covered five tasks from two repositories. The analysis used task-specific three-level quality ratings, the SUS score, preferences between the two tools used, and descriptive categorization of written reasons. Quality differences depended on the task. A11yHelper had higher mean scores than GitHub Copilot on form labeling and keyboard navigation in both evaluations, while GitHub Copilot matched or exceeded its means on the remaining tasks. Both scored higher than CodeA11y on the task requiring information distributed across a larger file. Alternative text remained difficult, with only two of 36 user-study artifacts rated Good. Mean SUS scores were 79.38 for GitHub Copilot, 76.46 for A11yHelper, and 59.58 for CodeA11y, preference counts followed the same ordering. The study contributes an integrated accessibility assistant and descriptive evidence of its task-specific strengths and limitations.