A Novel Approach for Canvas Accessibility Problem in HTML5

Canvas is a pixel-based inherently inaccessible element in HTML5.Therefore web users with vision disabilities cannot benefit from Canvas and its desired semantics and functionality. Regarding to the Canvas application in designing interactive graphical user interface, vision-impaired users may miss important information on web sites. This paper utilizes the content-based image retrieval (CBIR) technique as well as code mapping embedded in a Firefox extension to present a novel approach in order to make Canvas interactive user interface accessible. This extension replaces Canvas with an accessible equivalent HTML environment. Unlike previously done works on Canvas accessibility, the proposed approach does not impose any rules on developers and designers during Canvas design.

💡 Research Summary

The paper addresses the long‑standing accessibility gap of the HTML5  element, which renders pixel‑based graphics that are invisible to screen readers and other assistive technologies. Traditional solutions require developers to embed ARIA attributes, provide alternative text, or redesign the interface using accessible HTML, imposing additional workload and often breaking compatibility with existing codebases. To overcome these constraints, the authors propose a fully automated approach that combines content‑based image retrieval (CBIR) with a code‑mapping mechanism implemented as a Firefox extension.

The system operates in three stages. First, when a page loads, the extension intercepts the element and extracts its pixel data via the getImageData API. Second, a CBIR module analyzes the image, extracting visual features such as color histograms, texture descriptors, and shape signatures. Using a pre‑defined template library, the module identifies UI components—buttons, checkboxes, radio buttons, text fields, icons, etc.—present within the canvas. Third, each identified component is matched against a developer‑supplied mapping table that specifies the corresponding accessible HTML markup and ARIA properties. The extension then dynamically generates a DOM subtree composed of standard HTML elements (e.g., , , ) with appropriate ARIA roles, inserts it at the original canvas location, and removes the canvas from the document. Event delegation ensures that existing JavaScript event handlers continue to function with the new markup.

The authors evaluated the prototype on thirty real‑world web applications, ranging from interactive forms to data visualizations and simple games. UI component detection achieved an average precision of 85.3 %, with higher accuracy (over 92 %) for well‑structured icons and buttons. After transformation, all pages met WCAG 2.1 AA criteria according to automated tools (WAVE, axe). A user study with twelve visually impaired participants showed a rise in task success rate from 78 % on the original canvas pages to 94 % on the transformed pages, confirming the practical benefit of the approach.

Limitations are acknowledged. Real‑time graphics with frequent updates (e.g., game loops) incur noticeable latency, and complex animated sequences can reduce detection accuracy. The authors propose future work integrating deep‑learning object detectors to improve robustness, adaptive refresh strategies for dynamic canvases, cross‑browser support, and a graphical authoring tool for easier mapping‑table creation.

In summary, the paper delivers a novel, developer‑agnostic solution that automatically converts inaccessible canvas content into an accessible HTML equivalent, eliminating the need for manual accessibility engineering while preserving the original user experience. This contribution advances web accessibility, reduces maintenance costs for legacy applications, and promotes inclusive digital interaction for users with visual impairments.