Automatic Photo Adjustment Using Deep Neural Networks

Photo retouching enables photographers to invoke dramatic visual impressions by artistically enhancing their photos through stylistic color and tone adjustments. However, it is also a time-consuming and challenging task that requires advanced skills beyond the abilities of casual photographers. Using an automated algorithm is an appealing alternative to manual work but such an algorithm faces many hurdles. Many photographic styles rely on subtle adjustments that depend on the image content and even its semantics. Further, these adjustments are often spatially varying. Because of these characteristics, existing automatic algorithms are still limited and cover only a subset of these challenges. Recently, deep machine learning has shown unique abilities to address hard problems that resisted machine algorithms for long. This motivated us to explore the use of deep learning in the context of photo editing. In this paper, we explain how to formulate the automatic photo adjustment problem in a way suitable for this approach. We also introduce an image descriptor that accounts for the local semantics of an image. Our experiments demonstrate that our deep learning formulation applied using these descriptors successfully capture sophisticated photographic styles. In particular and unlike previous techniques, it can model local adjustments that depend on the image semantics. We show on several examples that this yields results that are qualitatively and quantitatively better than previous work.

💡 Research Summary

The paper tackles the problem of automatically applying artistic photo adjustments—such as stylized color and tone changes—to new images by learning from exemplar pairs of before‑and‑after photographs. Recognizing that many existing automatic methods operate globally and ignore image content, the authors formulate photo enhancement as a pixel‑wise regression problem and solve it with a deep neural network (DNN). Their key innovation lies in the design of a rich feature descriptor for each pixel that combines three levels of information: (1) the raw color of the pixel, (2) a contextual descriptor that captures the semantics of a large surrounding region, and (3) global image statistics. Semantic context is obtained by running a pretrained scene‑understanding model to assign class labels (e.g., sky, human, vehicle) to every pixel; these labels are then aggregated over multiple spatial scales using average pooling, producing a compact yet expressive representation of the local scene.

To handle high‑frequency color variations without forcing the network to model them directly, the authors factor the mapping into a color basis vector V(c) and a spatially varying transform matrix Φ(Θ, x). In the CIE‑Lab space, V(c) contains either the linear basis