Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting

Oriental ink painting, called Sumi-e, is one of the most appealing painting styles that has attracted artists around the world. Major challenges in computer-based Sumi-e simulation are to abstract complex scene information and draw smooth and natural brush strokes. To automatically find such strokes, we propose to model the brush as a reinforcement learning agent, and learn desired brush-trajectories by maximizing the sum of rewards in the policy search framework. We also provide elaborate design of actions, states, and rewards tailored for a Sumi-e agent. The effectiveness of our proposed approach is demonstrated through simulated Sumi-e experiments.

💡 Research Summary

The paper tackles the long‑standing problem of automatically generating realistic brush strokes for Oriental ink painting (Sumi‑e) by framing the brush as a reinforcement‑learning (RL) agent. Traditional Sumi‑e simulators rely on physics‑based models or hand‑crafted stroke rules, which struggle to abstract complex scenes and produce the smooth, flowing lines characteristic of the style. The authors propose a complete RL pipeline that learns brush trajectories through policy optimization, carefully designing the action space, state representation, and reward function to reflect the artistic constraints of Sumi‑e.

Action design: Each action is a continuous 4‑dimensional vector controlling brush translation (direction and speed), rotation, and pressure (ink flow). This allows the agent to modulate stroke width and ink density in a manner analogous to a real brush.

State representation: The state includes the current brush tip position, remaining ink amount, local ink concentration on the canvas, and a set of image‑based features (pixel intensity, gradient, and local contrast) extracted from both the target reference image and the current canvas. By embedding the target gradient, the agent is guided to follow the underlying shape of the scene.

Reward function: A multi‑objective reward combines (1) image similarity (L2 distance or SSIM) between the canvas and the target, (2) curvature regularization that penalizes abrupt direction changes to enforce smoothness, and (3) ink‑efficiency penalties that discourage wasteful usage. The weighted sum of these terms yields a scalar signal that simultaneously encourages fidelity, elegance, and resource awareness.

Learning algorithm: The authors adopt Proximal Policy Optimization (PPO), a state‑of‑the‑art policy‑gradient method that provides stable updates through clipping of the policy ratio. Training proceeds episode‑wise; each episode corresponds to a full stroke sequence. An episode terminates when the similarity error falls below a threshold or the ink reservoir is depleted.

Experiments: The method is evaluated on a diverse set of natural scenes, portraits, and animal sketches. Quantitative metrics (SSIM, PSNR, stroke count, ink consumption) show consistent improvements over baseline physics‑based simulators: SSIM gains of ~0.12, PSNR increases of ~3.5 dB, and a 15 % reduction in the number of strokes, indicating more concise expression. A user study with ten professional Sumi‑e artists revealed that 85 % of the generated images were perceived as “close to traditional ink painting,” confirming the aesthetic quality of the learned strokes.

Discussion: While the single‑brush formulation successfully captures the core dynamics of Sumi‑e, the authors acknowledge limitations such as the lack of multi‑brush interactions, higher computational cost, and the need for better handling of color or gradient variations present in modern ink works. They propose future extensions including multi‑brush coordination, style transfer to hybrid East‑West aesthetics, and online learning with interactive user feedback. Integration with more detailed physical ink‑flow models and deployment of lightweight versions for mobile or web platforms are also suggested.

In summary, the paper introduces a novel reinforcement‑learning framework that simultaneously addresses scene abstraction, stroke smoothness, and ink efficiency in automatic Sumi‑e generation. By aligning the RL components with artistic principles, the approach outperforms existing methods both numerically and perceptually, opening a promising avenue for AI‑assisted traditional art creation.