What Can Artificial Intelligence Tell Us About Fine Art?

Dissecting the interrelationship between aesthetics, sentiment and memorability reveals a complex interplay between the emotional and artistic quality.

Which may explain paintings of wide-eyed children on black velvet but will hopefully not encourage painters to find inspiration strictly from the data....JL


Michelle Hampson reports in IEEE Spectrum:

Beautiful paintings are not necessarily memorable. Researchers are using machine-learning algorithms to explore the relationship between aesthetics, sentimental value, and memorability of art. Models that reflect human preferences uncover subtleties that influence human judgment when it comes to art. The model found bold and intense paintings are the most pleasing. But the factors that make art more attractive, such as color harmony and vividness, negatively correlate with the sentimental value of an image.Abstract paintings are memorable, but have a low aesthetic score; landscapes are pleasing to the eye, but not memorable.
Whether it’s the enigmatic playfulness of Mona Lisa’s smile or the swirling soft colors of a Monet painting, there are qualities of fine art that attract audiences, like a moth to a flame. What is it about these pieces that has captivated people throughout centuries? Researchers are now using machine-learning algorithms to tease apart these intricacies and explore the relationship between the aesthetics, sentimental value, and memorability of fine art.
Eva Cetinic is an art enthusiast and researcher at the Rudjer Boskovic Institute in Croatia. While she believes that art is indescribable in many ways, she wanted to challenge her own perspective by exploring how machine learning might quantify art. “The rise of artificial intelligence forces us to re-think what values are specifically human, and the understanding of art is a particularly fruitful playground for this kind of investigation,” she explains.
To start, Cetinic and her colleagues analyzed more than 100,000 images from WikiArt. Their results, published 5 June in IEEE Access, hint at common themes of what we find beautiful and captivating.
The researchers took several existing models that are respectively trained to analyze the aesthetics, sentimental value, and memorability of photos, and modified the models to be more applicable for fine art. They then compared the predictive scores by the models to scores given by people in other studies, whereby participants evaluated pieces of fine art for factors such as aesthetic quality, beauty, color, content, and composition. From there, the researchers selected the predictive model that best matched human preferences for each category.
By selecting models that closely reflect human preferences, AI may be able to uncover the subtleties that influence human judgment when it comes to art.

Unsurprisingly, the model chosen to analyze aesthetics found that bold and intense paintings are the most pleasing, while dim and dull paintings are less so. But the factors that make art more attractive to the AI eye, such as color harmony (meaning if the colors go nicely together) and vividness, actually negatively correlate with the sentimental value of an image. So rather than color, the model found that a person’s emotional response to a piece is more closely tied to things such as flowers and smiling people, while outdoor scenes or sad or fearful faces are less sentimental.
Perhaps reflective of human nature, the models found nudity particularly memorable. Intriguingly, abstract images were also found to be memorable, which the authors say may be due to the absence of objects that we recognize. Because we rarely encounter the visual stimuli seen in abstract paintings, the image may draw the viewer’s attention more than a painting containing an object we are familiar with.

Photo: Eva Cetinic

These 100 artworks from the WikiArt collections scored the highest for predicted memorability.

So how do all of these factors relate to one another? Apparently, beautiful paintings are not necessarily memorable. In fact, while there was a correlation in the models’ predictions between what’s considered aesthetically pleasing and what’s considered sentimental, both of these factors negatively correlated with the memorability of a painting. For example, abstract paintings are memorable, but have a low aesthetic score; in contrast, landscapes are pleasing to the eye, but not memorable.
The researchers also analyzed the data by artist and era. Ironically, William Turner, the artist who produced the most visually appealing pieces of work, also scored the lowest in terms of memorability. Ah, the woes of being an artist.
Cetinic notes that the model for sentimental value awarded the only female artist in this subset, Frida Kahlo, with a significantly higher score than the other artists. “This might be because Kahlo’s paintings often include features which the model learned to identify as highly positive, such as color vividness and flowers,” she says. “However, a human observer familiar with Kahlo’s work knows that there is actually a lot of pain in her paintings, and that a shallow understanding of the sentiment of particular motifs cannot grasp the emotional complexity of her artistic expression. This is an example which shows the current limitations of this approach but also indicates new possible directions for future research.”