StereoMath: An Accessible and Musical Equation Editor

For blind and low-vision (BLV) individuals, digital math communication is uniquely difficult due to the lack of accessible tools. Currently, the state of the art is either code-based, like LaTeX, or WYSIWYG, like visual editors. However, both paradigms view math communication as primarily a visual typesetting problem, and may be accessible but difficult to use. In this paper, we present an equation editor that is built from the ground up with BLV accessibility in mind. Specifically, we notice that two of the biggest barriers with current technology are the high cognitive load and the lack of spatial relationships. Thus, we build an editor that uses spatial audio cues, muscle memory, tones, and more intuitive navigation to properly contextualize math equations. We discuss how this new paradigm can enable new levels of math communication, engagement, and literacy. Finally, we discuss natural next steps.

💡 Research Summary

StereoMath is a web‑based equation editor designed from the ground up for blind and low‑vision (BLV) users. The authors identify two major barriers in existing tools: high cognitive load and the loss of spatial relationships that are essential for understanding mathematics. To address these, StereoMath combines three auditory modalities—stereo (left/right) audio, pitch (up/down) modulation, and earcons (short non‑speech sound icons)—with a keyboard‑driven spatial navigation scheme that leverages the user’s own keyboard as a tactile map of the equation layout.

The system maintains two internal representations of an equation: a tree where operators are internal nodes and operands are leaves, and a 2‑D grid (table) that maps the visual layout into rows and columns. The tree enables conventional pre‑order traversal for linear reading, while the grid supports “row mode” and “column mode” navigation, allowing the user to move a virtual cursor horizontally or vertically using arrow keys. Each key press is mapped to a physical location on the keyboard; pressing a key in the top‑right corner, for example, selects the element in the corresponding top‑right cell of the equation grid. When an element is selected, a short melodic phrase indicates its position, and a spoken read‑out (either MathSpeak or a custom “IntuitiveSpeak”) provides the verbal description.

Spatial sound is generated by a custom backend that uses Piper TTS to synthesize raw speech, then applies binaural panning (left/right) and pitch shifting (high/low) via Tone.js. This real‑time audio processing makes the user hear the “leftmost” term in the left speaker and the “topmost” term at a higher pitch, thereby conveying two dimensions of spatial information without visual cues. Earcons replace verbose verbal tokens such as “lparen” and “rparen” with brief tonal cues, reducing working‑memory load and increasing information density. Pseudo‑parentheses are inserted internally to mark the start and end of nested nodes, aiding disambiguation without appearing in the final LaTeX output.

The editor also offers a high‑contrast UI for users with residual vision, customizable settings (e.g., turning off spatial audio for unilateral hearing loss, adjusting earcon volume, importing custom keyboard layouts), and full LaTeX generation for downstream use. A “loopback buffer” lets users replay the current node or the entire expression in either MathSpeak or the shorter IntuitiveSpeak, supporting both precise and rapid comprehension. Code folding and a “read‑friendly mode” further reduce visual clutter for screen‑reader users.

The development process followed a mixed‑ability co‑design methodology: a blind researcher with 23 years of screen‑reader experience and a sighted researcher iteratively prototyped, tested, and refined the system over several months. This collaboration yielded novel solutions such as the spatial keyboard navigation and music‑inspired earcons.

Limitations noted by the authors include the lack of large‑scale quantitative user studies at this stage and the dependence on auditory perception, which may vary across users. Future work will involve systematic user evaluations, personalization of spatial audio parameters, and integration of additional modalities such as haptic feedback. The authors envision extending StereoMath into a full‑featured mathematical workbench, potentially supporting proof authoring, graphing, and collaborative editing.

In summary, StereoMath reimagines equation editing as an auditory‑spatial experience, offering BLV users a more intuitive, low‑cognitive‑load way to read, write, and understand mathematical notation, and sets a foundation for broader multimodal accessibility research in STEM domains.