On a constant quest for inspiration, designers can become more effective with tools that facilitate their creative process and let them overcome design fixation. This paper explores the practicality of applying neural style transfer as an emerging design tool for generating creative digital content. To this aim, the present work explores a well-documented neural style transfer algorithm (Johnson 2016) in four experiments on four relevant visual parameters: number of iterations, learning rate, total variation, content vs. style weight. The results allow a pragmatic recommendation of parameter configuration (number of iterations: 200 to 300, learning rate: 2e-1 to 4e-1, total variation: 1e-4 to 1e-8, content weights vs. style weights: 50:100 to 200:100) that saves extensive experimentation time and lowers the technical entry barrier. With this rule-of-thumb insight, visual designers can effectively apply deep learning to create artistic visual variations of digital content. This could enable designers to leverage AI for creating design works as state-of-the-art.
The search for inspiration represents an essential part of visual design. Yet, both design students and professional designers have been observed to consider only a limited range of inspirational approaches (Gonçalves, Cardoso, and Badke-Schaub 2014). One method to overcome this limitation is to consider external stimuli in the idea generation stage.
Designers commonly employ this method in their design process, and many studies have confirmed its positive effects. Nevertheless, the exposure to external stimuli also leads to design fixation, the narrowing of the creative solution space (Jansson and Smith 1991).
Therefore, it is critical to find the right balance between providing external stimulation on the one side and avoiding design fixation on the other side. To reach this goal, the current work proposes an approach based on artificial intelligence, or AI. The critical factor to avoid design fixation is that the AI element does not generate new solutions at random, but only based on the designer’s decision on new style information. Hence, the AI element is not used as a solution generator to replace human creativity but rather as a facilitator for imagining new visual outcomes. This catalyst function of AI may effectively support the creative process by harnessing the latest advancement of artificial intelligence research in the field of deep learning. Particularly interesting for designers and artists, an emerging deep learning topic has become the simulation of human creativity by so-called generative models.
An example of generative models is neural style transfer, i.e. the process of merging a visual input with another one. This algorithm gained public attention by an experiment of Hollywood’s film-making industry in early 2017: Kirsten Stewart, protagonist of the popular US series “The Twilight Saga”, directed the movie “Come Swim” applying neural style transfer and co-authored a research paper that described how this AI algorithm produced the special effects (Joshi, Stewart, and Shapiro 2017).
Technically, this new algorithm built on substantial progress in the development of convolutional neural networks (CNNs) and generative adversarial networks (GANs). Gatys, Ecker, and Bethge (2015) introduced this algorithm to show how the artistic style of painters can be transferred to another image. The neural style algorithm extracts the semantic information of the input image, learns the color and texture information in the style image, and then renders the semantic content of the input image in the color and texture of the style image (Gatys, Ecker, and Bethge 2016).
Designers can create new inspiration by viewing variations that are generated with styles outside their consideration. For example, non-artistic style transfer generates new styles by tiling images of everyday photo motifs like guitars or faces, and succesfully transfers them to the content image (Kenstler 2017). In early conceptual stages, a silhouette can be filled by a style image with neural style transfer that aligns with the silhouette outline (Ramea 2017).
Originally, neural style transfer was demonstrated with common photo motifs like houses or landscapes (Gatys, Ecker, and Bethge 2016). Neural style transfer has later been applied to doodles (Champandard 2016), videos (Huang et al. 2017), artistic improvisation (Choi 2018), and fashion (Jiang and Fu 2017;Zhu et al. 2017).
Recent developments of neural style transfer include using pretrained models for stylization in so-called feedforward networks (Chen and Schmidt 2016). One such approach (Johnson, Alahi, and Fei-Fei 2016) leverages the use of the pretrained models to calculate losses on high-level features (so-called perceptual losses) instead of per-pixel losses.
Based on the advancement of neural style transfer, it becomes interesting from a design perspective whether it can replace effortful manual design tasks. For example, designers must often create visual variations of a motif like a person portrait, e.g. for album and book covers or posters.
Although the exploration of AI technology looks promising, it is relatively inaccessible to designers. First, it requires the purchase of an advanced graphics card (GPU) that is not standard equipment in desktop or notebook computers. Moreover, a typical AI framework requires the installation of over 50 programs that all must be compatible to each other, including GPU drivers (e.g. CUDA, CUDNN), compilers (e.g. gcc, g++), deep learning libraries (Numpy, Skipy, Scikit-Learn, Hdf5), and loss networks (e.g. ResNet50).
In addition to this technical setup complexity, a designer is confronted with understanding an AI framework that conceptually offers not even remotely a connection with any other design tools or techniques. Therefore, the present work aims at alleviating this accessibility problem for AI to provide a practical guide to the research question:
How can designers configure a neural style transfer algorithm to produce
This content is AI-processed based on open access ArXiv data.
