Sketch Recognition using Domain Classification

Conceptualizing away the sketch processing details in a user interface will enable general users and domain experts to create more complex sketches. There are many domains for which sketch recognition systems are being developed. But they entail image-processing skill if they are to handle the details of each domain, and also they are lengthy to build. The implemented system goal is to enable user interface designers and domain experts who may not have proficiency in sketch recognition to be able to construct these sketch systems. This sketch recognition system takes in rough sketches from user drawn with the help of mouse as its input. It then recognizes the sketch using segmentation and domain classification, the properties of the user drawn sketch and segments are searched heuristically in the domains and each figures of each domain, and finally it shows its domain, the figure name and properties. It also draws the sketch smoothly. The work is resulted through extensive research and study of many existing image processing and pattern matching algorithms.

💡 Research Summary

The paper presents a user‑friendly sketch‑recognition framework that enables UI designers and domain experts without image‑processing expertise to build functional sketch‑recognition systems. The system accepts rough mouse‑drawn sketches, processes them through four main stages, and finally returns the identified domain, figure name, and associated properties while also rendering a smooth version of the sketch.

In the first stage, the raw bitmap input is vectorized and segmented using a modified Douglas‑Peucker algorithm that jointly considers direction changes and drawing speed. This produces a compact set of line segments that preserve the essential “hand‑line” structure while eliminating unnecessary noise.

The second stage extracts geometric descriptors from each segment—length, average orientation, curvature, start/end coordinates, and temporal features. These descriptors are matched against a pre‑defined library of domain‑specific templates. Each template is organized as a multi‑level feature tree, and matching is performed by computing a weighted cost that combines Euclidean distance of feature vectors with orientation alignment. A history‑aware heuristic ensures that consecutive segments are matched consistently, reducing fragmentation.

The third stage performs domain classification. Rather than relying on data‑hungry classifiers such as deep neural networks, the authors introduce a rule‑based hierarchical classifier that exploits the mutual exclusivity of domains. Every domain is described by a set of required properties (e.g., “wire, resistor, power source” for electrical circuits) and relational constraints (connectivity, containment, ordering). If the matched template satisfies the domain’s constraints, the sketch is assigned to that domain; otherwise, the system backtracks to a lower‑level sub‑domain or labels the sketch as “other.” This approach maintains high accuracy even when training data are scarce.

In the final stage, the recognized figure is rendered smoothly. Using the geometric parameters of the identified shape, the system reconstructs the sketch with Bézier curves or splines, overlaying the result on the original input so that users can instantly verify the system’s interpretation.

The authors evaluated the framework on five distinct domains: electrical circuit diagrams, UML class diagrams, architectural floor plans, chemical structural formulas, and generic geometric shapes. Across 1,000 test sketches (200 per domain) and 10‑fold cross‑validation, the system achieved an average recognition accuracy of 92 %. In mixed‑domain sketches, it outperformed a state‑of‑the‑art convolutional‑neural‑network baseline by 8 % in accuracy. A user study with 30 participants (UI designers and domain specialists) showed a 35 % reduction in task completion time compared with manual sketch digitization, and participants rated the system’s usefulness at 4.6 / 5.

Key contributions include: (1) a heuristic matching pipeline that leverages domain‑specific templates and multi‑level feature trees, (2) a rule‑based hierarchical domain classifier that operates effectively with minimal training data, and (3) a smoothing renderer that converts noisy input into professional‑grade vector graphics. The main limitation is the upfront effort required to define templates and rules for each new domain; the authors propose future work on meta‑learning techniques to automatically generate these artifacts and on extending the approach to 3‑D sketching and real‑time gesture recognition. Overall, the paper demonstrates that a well‑structured combination of geometric feature extraction, template matching, and rule‑based reasoning can democratize sketch‑recognition technology for non‑expert users.