Toward a Machine Bertin: Why Visualization Needs Design Principles for Machine Cognition

Visualization’s design knowledge-effectiveness rankings, encoding guidelines, color models, preattentive processing rules – derives from six decades of psychophysical studies of human vision. Yet vision-language models (VLMs) increasingly consume chart images in automated analysis pipelines, and a growing body of benchmark evidence indicates that this human-centered knowledge base does not straightforwardly transfer to machine audiences. Machines exhibit different encoding performance patterns, process images through patch-based tokenization rather than holistic perception, and fail on design patterns that pose no difficulty for humans-while occasionally succeeding where humans struggle. Current approaches address this gap primarily by bypassing vision entirely, converting charts to data tables or structured text. We argue that this response forecloses a more fundamental question: what visual representations would actually serve machine cognition well? This paper makes the case that the visualization field needs to investigate machine-oriented visual design as a distinct research problem. We synthesize evidence from VLM benchmarks, visual reasoning research, and visualization literacy studies to show that the human-machine perceptual divergence is qualitative, not merely quantitative, and critically examine the prevailing bypassing approach. We propose a conceptual distinction between human-oriented and machine-oriented visualization-not as an engineering architecture but as a recognition that different audiences may require fundamentally different design foundations-and outline a research agenda for developing the empirical foundations the field currently lacks: the beginnings of a “machine Bertin” to complement the human-centered knowledge the field already possesses.

💡 Research Summary

The paper argues that the body of visualization design knowledge—Bertin’s visual variables, Cleveland‑McGill effectiveness rankings, pre‑attentive processing rules, Gestalt principles, and color‑science guidelines—has been built exclusively on human perceptual research over the past six decades. With the rise of vision‑language models (VLMs) such as GPT‑4o, Claude 3.5, and other transformer‑based vision systems, charts are now being consumed directly by machines in automated pipelines. Empirical evidence from a growing set of benchmarks (ChartQA, CharXiv, ChartMuseum, EncQA, ChartInsights, etc.) shows that VLMs perform substantially worse on human‑designed visualizations, often exhibiting a 30 %+ accuracy gap and error patterns that are qualitatively different from human mistakes. Studies that replicate classic human perception experiments (e.g., Cleveland‑McGill) with Vision Transformers reveal divergent effectiveness rankings, confirming that the human‑machine perceptual divergence is not merely quantitative but structural.

Current practice addresses this gap by “bypassing vision”: converting charts into tables, structured text, or declarative specifications (e.g., DePlot, MatCha, UniChart, Vega‑Lite). While this improves immediate performance, it sidesteps the deeper question of whether visual representations can be engineered to serve machine cognition directly. The authors contend that this bypassing approach forecloses a research agenda that could inform both how machines consume visual data and how AI agents generate visualizations intended for other AI agents.

The paper proposes a conceptual split between human‑oriented visualization (design for human perception) and machine‑oriented visualization (design for machine cognition). It calls for the development of a “machine Bertin”—a set of empirically derived design principles tailored to the processing mechanisms of VLMs. The proposed research roadmap includes: (1) systematic mapping of VLM tokenization and attention to traditional visual variables; (2) derivation of machine‑friendly visual design rules (e.g., emphasizing shape or grid regularity over color contrast); (3) hybrid visual‑text interfaces that provide meta‑information alongside pixels; (4) standards for AI‑to‑AI visual communication within pipelines; and (5) new benchmarks that evaluate both accuracy and the nature of error patterns under machine‑oriented designs.

By highlighting the historical contingency of visualization’s human‑centric foundations and demonstrating that extending the audience to machines creates a knowledge gap, the paper makes three contributions: a theoretical analysis of the human‑centric knowledge base, a critique of the bypassing strategy, and a forward‑looking agenda for building machine‑oriented visualization theory. The authors argue that without such a shift, future AI‑augmented analytics, automated reporting, and human‑AI collaborative systems will continue to rely on suboptimal visual representations, limiting the potential of visual information to enhance machine reasoning.