A Quantitative Approach to Painting Styles

A Quan titativ e Approac h to P ain ting St yles Vilson Vieira 1 , Renato F abbri 1 , Da vid Sbrissa 1 , Luciano da F on toura Costa 1 , Gonzalo T ra vieso 1 1 Instituto de F ´ ısica de S˜ ao Carlos, Univ ersidade de S˜ ao P aulo (IFSC/USP) E-mail: vilsonvieira@usp.br , fabbri@usp.br , davidsbrissa@hotmail.com , ldfcosta@gmail.com , gonzalo@ifsc.usp.br Abstract. This researc h extends a metho d previously applied to m usic and philosoph y [1], representing the ev olution of art as a time-series where relations lik e diale ctics are measured quan titativ ely . F or that, a corpus of pain tings of 12 w ell-known artists from baroque and mo dern art is analyzed. A set of 93 features is extracted and the features which most contributed to the classification of painters are selected. The pro jection space obtained provides the basis to the analysis of measurements. This quan titativ e measures underlie rev ealing observ ations about the evolution of pain ting st yles, specially when compared with other h umanity fields already analyzed: while m usic ev olv ed along a master-apprentice tradition (high dialectics) and philosophy b y opp osition, pain ting presen ts another pattern: constan t increasing sk ewness, lo w opp osition b et w een members of the same mo v emen t and opp osition p eaks in the transition betw een mo vemen ts. Differences betw een baro que and mo dern mov ements are also observ ed in the pro jected “pain ting space”: while baroque paintings are presen ted as an ov erlapp ed cluster, the mo dern paintings present minor o v e rlapping and are disposed more widely in the pro jection than the baro que coun terparts. This finding suggests that baro que painters shared aesthetics while mo dern painters tend to “break rules” and dev elop their o wn st yle. P A CS num b ers: 05.10.-a, 89.65.-s Keywor ds: P attern recognition, arts, painting, feature extraction, creativity A Quantitative Appr o ach to Painting Styles 2 1. In tro duction P ainting classification is a common field of interest for applications suc h as pain ter iden tification — e.g. assessing the authen ticit y of a giv en art w ork — st yle classification, pain tings data base search and more recently , automatic aesthetic judgment in computational creativit y applications. Determining the b est features for painting style c haracterization is a complex task on its own. Man y studies [2, 3, 4, 5] applied image pro cessing to feature extraction for pain ter and art mov emen ts identification. Mano vich [6, 7, 8] uses features lik e entrop y , brigh tness and saturation to map paintings and general images in to a 2-dimensional space and, in this w a y , to visualize the difference b et w een pain ters. There are also man y related w orks dealing on feature selection for painting classification. P enousal et al. [9] use features based on aesthetic criteria estimated b y image complexity while Zujovic et al. [10] ev aluate a large set of features that most contribute to classification. This study also analyses a set of features whic h most con tribute to the classification of pain tings. Although, in contrast with previous w orks, it go es forward: the historic ev olution of painting styles is analyzed by means of geometric measures in the feature space. Those measures – opp osition , skewness and diale ctics – are central while discussing human history . How ever, such discussions are common only at humanities fields like Philosoph y and those quantitativ e measures are suggested to do not surpass but contribute in this understanding of h uman history . T o create the feature space, a set of 93 features is extracted from 240 images of 12 w ell-known pain ters. The first six pain ters of this group represen t the baro que mo vemen t while the remaining six represen t the mo dern art p erio d. A feature selection pro cess yields the pair of features which most con tributed for the classification. Similar results using LD A (Linear Discriminant Analysis) analysis are obtained, whic h reinforces the feature selection. After feature selection, a centroid for eac h group of pain tings is calculated which defines a pr ototyp e : a representativ e work-piece for the resp ective cluster. The set of all prototypes following a c hronological order defines a time-series where the main purp ose of this study is p erformed: the quantitativ e analysis of the historical ev olution of art mo v ements. Extending a metho d already applied to music and philosophy [1], opp osition , skewness and diale ctics measuremen ts are taken. These concepts are cen tral in philosoph y — e.g. philosophers from an tiquit y lik e Aristotle and Plato developed their ideas using the dialectics metho d while it is also found in mo dern works lik e Hegelian and Marxist dialectics — and humanistic fields, how ev er lacks studies from a quan titative p ersp ectiv e. [11] Represent ed as geometric measures, these concepts rev eal in teresting results and patterns. Mo dern pain tings groups show minor sup erp osition when compared with baroque coun terparts suggesting the independence in style found historically in mo dernists and strong influence of shared painting tec hniques found in baro que painters. Dialectics and opp osition v alues presen ted a p eak in the transition b et w een baro que and mo dern p erio ds — as exp ected considering history of art — A Quantitative Appr o ach to Painting Styles 3 T able 1. P ainters ordered chronologically with the artistic style they represents. artists Remark able Styles/Mo vemen ts Cara v aggio Baro que, Renaissance F rans Hals Baro que, Dutc h Golden Age Nicolas P oussin Baro que, Classicism Diego V el´ azquez Baro que Rem brandt Baro que, Dutc h Golden Age, Realism Johannes V ermeer Baro que, Dutc h Golden Age Vincen t v an Gogh P ost-Impressionism W assily Kandinsky Expressionism, Abstract art Henri Matisse Mo dernism, Impressionism P ablo Picass o Cubism Joan Mir´ o Surrealism, Dada Jac kson P ollo c k Abstract expressionism with decreasing v alues in the b eginning of each p erio d. Sk ewness index is presen ted with oscillating but increasing v alues during all the time-series, suggesting a constant inno v ation through art mo v ements. These results presen t an interesting counterpart with previous results in philosoph y — where opp osition is strong in almost entire time- series — and in music — where the dialectics is remark able [1]. The study starts describing the corpus of paintings used and a review of b oth aesthetic and historic facts regarding baro que and mo dern mov ements (Section 2). The image pro cessing steps used to extract features from these pain tings are presen ted follo wed b y the feature selection. The results are them discussed in Section 3 with basis on geometric measurements in the pro jected feature space – considering the most clustered pro jection and LDA comp onents. 2. Mo deling painting mo v ements 2.1. Painting c orpus A group of 12 well-kno wn painters is selected to represent artistic st yles or mov emen ts from baro que to modernism. Six painters are c hosen to represent eac h of these mo vemen ts. The group is presen ted in T able 1 together with their more representativ e st yle, in c hronological order. It is kno wn that painters like Picasso co vered more than one st yle during his life. Although, only the most remark able st yle is selected intending to well c haracterize the painter b y means of this sp ecific p erio d or mo vemen t. F or each pain ter, 20 raw images are considered from the database of public images organized b y Wikip edia. Examples of selected paintings titles and their resp ective creation year are listed in T able 2 ‡ ‡ The source co de together with all the 240 raw images are a v ailable online at http://github.com/ automata/ana- pintores. A Quantitative Appr o ach to Painting Styles 4 T able 2. Some of the 240 selected paintings and their resp ective author and year of creation. P ain ter P ain ting title Y ear Cara v aggio Musicians 1595 Judith Beheading Holofernes 1598 Da vid with the Head of Goliath 1610 F rans Hals P ortrait of an unkno wn woman 1618/20 P ortrait of P aulus v an Beresteyn 1620s P ortrait of Stephan us Geeraerdts 1648/50 Nicolas P oussin V en us and Adonis 1624 Cephalus and Aurora 1627 Acis and Galatea 1629 Diego V el´ azquez Three m usicians 1617/18 The Lunc h 1618 La m ulatto 1620 Rem brandt The Sp ectacles-p edlar (Sigh t) 1624/25 The Three Singers (Hearing) 1624/25 Balaam and the Ass 1626 Johannes V ermeer The Milkmaid 1658 The Astronomer 1668 Girl with a P earl Earring 1665 Vincen t v an Gogh Starry Nigh t Ov er the Rhone 1888 The Starry Nigh t 1889 Self-P ortrait with Stra w Hat 1887/88 W assily Kandinsky On White I I 1923 Comp osition X 1939 P oin ts 1920 Henri Matisse Self-P ortrait in a Strip ed T-shirt 1906 P ortrait of Madame Matisse 1905 The Dance (first v ersion) 1909 P ablo Picass o Les Demoiselles d’Avignon 1907 Guernica 1937 Dora Maar au Chat 1941 Joan Mir´ o The F arm 1921/22 The Tilled Field 1923/24 Bleu I I 1961 Jac kson P ollo c k No. 5 1948 Autumn Rh ythm 1950 Blue P oles 1952 A Quantitative Appr o ach to Painting Styles 5 It is interesting to raise some historical and aesthetic c haracteristics from baro que and mo dern mov emen ts b efore entering the quan titative analysis in Section 3 where those h yp othesis are further discussed. Baro que is marked by tradition, a desire to p ortrait the truth (found in Cara v aggio, F rans Hals and V el´ azquez), the b eaut y (Poussin, V ermeer), the nature and the sacred (Carav aggio, Rembrandt). A remark able use of ligh t con trast (as in the “ chiar oscur o ” tec hnique mastered by Cara v aggio), disregarding for simple equilibrium in comp osition and preference for complex opp ositions, b oth comp ound aesthetic c haracteristics which baroque artists used to represent their view of nature. The transmission of those techniques from one painter to another is common in baroque. Mo dernists, on the other hand, did not follo w “rules”. Each modern pain ter emplo yed or created new wa ys to represen t nature. As noted b y Gom bric h [12]: “[they] cra ved for an art that do es not consists of tricks that could b e learn, for a style that is not a mere st yle, but something strong and pow erful lik e the human passion”. V an Gogh pursued this artistic trend in his in tense use of colors and the caricature aspect of his pain tings. Paul Gauguin searched for “primitive” in his paintings. Others, like Seurat, applied physical prop erties of the chromatic vision and started pain ting the nature like a collection of color p oin ts, and ended creating the p ointillism. Mo dernists created a new style for eac h of their experiments using their own techniques to represent a nature outside of the domains already cov ered by their predecessors. 2.2. Image pr o c essing All 240 images are re-sized to 800x800 pixels and cropped to consider a region p ositioned in the same co ordinates and with same asp ect of b oth original paintings, and pre- pro cessed b y applying histogram equalization and median filtering with a 3-size windo w. F eature extraction algorithms are applied to colored, gray-scale or binary v ersions of images as necessary (e.g. con vex-h ull used a binary image, whereas Haralick texture used the gra y-scale image and SLIC segmentation analysis is applied to color images). Curv ature measurements are extracted from segments of paintings identified by the SLIC segmen tation metho d [13] as presented in Figure 2. The whole pro cess is represented sc hematically in Figure 1 and cov ers all the steps from image pro cessing through measuremen ts, discussed in the following sections. 2.3. Extr acte d fe atur es T o create a p ainting sp ac e a num b er of distinct features extracted b y computational metho ds from raw images of the paintings is considered. The features are related with aesthetics characteristics and aim to quan tify prop erties w ell-known b y art critics. All the features are summarized in T able 3 and detailed, group ed in classes, in the follo wing list. Gener al shap e fe atur es : after image segmen tation, a num b er of shape descriptors are calculated for eac h segmen t, represen ted as a binary matrix. Perimeter is measured as pixel-length of the segmen t con tour. A r e a is estimated counting the A Quantitative Appr o ach to Painting Styles 6 Raw Images Processed Images Pre-processing Shapes Segmentation Feature Vectors Feature Extraction and Normalization Best Features Prototypes Time-series and Measured Values Measurements Calculation Components Confusion Matrix LDA Validation Prototypes Calculation Feature Selection LDA Figure 1. A summary of all steps from image pro cessing through feature extraction through time series and measurements calculation (skewness, opp osition and dialectics). n umber of pixels representing the segment. A conv ex-h ull of the segment is used to calculate the c onvex ar e a and its ratio to the original segment area. The numb er of c onstituent se gments for each pain ting is also considered as a descriptor. Simple c omplexity fe atur es : Cir cularity rev eals how m uch a shap e remembers a circle and is obtained b y the ratio b et w een p erimeter and area of the segmen t. T o estimate image complexity , a num b er of entr opy measures of its energy (squared FFT co efficien ts) are computed — listed in the first quarter of T able 3. T ogether with en tropy , a more specific family of measuremen ts is considered for texture c haracterization: the 11 Har alick textur e fe atur es [14] are calculated for this purp ose. Curvatur e : this descriptor has an in teresting biological motiv ation related to the h uman visual system — e.g. ob ject recognition is related to the iden tification of corners and high curv ature p oints [15]. Those p oints hav e more information ab out ob ject shap e than straigh t lines or smo oth curves. In this sense, curv ature is w ell suited for the characterization of the considered pain tings. Curv ature k ( t ) of a parametric curv e c ( t ) = ( x ( t ) , y ( t )) is defined as: k ( t ) = ˙ x ( t ) ¨ y ( t ) − ˙ y ( t ) ¨ x ( t ) ( ˙ x ( t ) 2 + ˙ y ( t ) 2 ) 3 2 (1) A Quantitative Appr o ach to Painting Styles 7 T able 3. Extracted features. Num b er of features F eatures 4 Energy of the whole image 4 Energy µ of image ro ws 4 Energy σ of image rows 4 Energy µ of image columns 4 Energy σ of image rows 4 Energy cen troids of image ro ws 4 Energy cen troids of image columns 4 Energy µ of ro ws and columns 4 Energy σ of rows and columns 1 µ of lo cal en trop y (5-size windo w) 1 µ of lo cal en trop y (50-size windo w) 4 Angular second momen t 4 Con trast 4 Correlation 4 Sum of squares: v ariance 4 In v erse difference moment 4 Sum a v erage 4 Sum v ariance 4 Sum en trop y 4 En trop y 4 Difference a v erage 4 Difference en trop y 2 µ of distance b et w een curv ature p eaks 2 σ of distance b etw een curv ature p eaks 1 µ of n um b er of curv ature p eaks 1 µ of segmen ts p erimeter 1 µ of segmen ts area 1 µ of circularit y ( P er . 2 / Ar ea ) 1 µ of n um b er of segments 1 µ of con v ex-h ull area 1 µ of con v ex-h ull and original areas ratio 93 T otal of extracted features b eing t the arc-length parameter and ˙ x ( t ), ˙ y ( t ), ¨ x ( t ) and ¨ y ( t ) are respectively the first and second order deriv ativ es of x ( t ) and y ( t ). Those deriv atives are obtained through F ourier transform and conv olution theorem: ˙ x = = − 1 (2 π iω X ( ω )) (2) ˙ y = = − 1 (2 π iω Y ( ω )) (3) ¨ x = = − 1 ( − (2 π ω ) 2 X ( ω )) (4) ¨ y = = − 1 ( − (2 π ω ) 2 Y ( ω )) (5) A Quantitative Appr o ach to Painting Styles 8 where = − 1 is the inv erse F ourier transform, X and Y are the F ourier transform of x and y respectively , ω is the angular frequency and i is the imaginary unit (see Figure 2). The corresp onding features are calculated from the curv ature data: the me an and standar d deviation of data, the numb er of p e aks and the distanc e (geometric and in pixels) b etw een p eaks. It is important to note that a p e ak is defined as a high curv ature p oin t. A p oint a is considered a p eak if its curv ature k ( a ) satisfies the following criteria: k ( a ) > k ( a − 1) (6) k ( a ) > k ( a + 1) (7) k ( a ) > τ (8) b eing τ the corresp onding threshold defined as median ( k ) γ (9) where γ is a factor obtained empirically as v alues which reveal the desired level of curv ature detail. 2.4. Me asur ements N features define a N -dimensional space, also called p ainting sp ac e where the follo wing measuremen ts are calculated [1]. F or simplification, a protot yp e ~ p i is defined for eac h class C p . Eac h prototype summarizes a painting class, b eing its c entr oid : ~ p i = 1 N p P N p i =1 ~ f i calculated in the pro jected space as well. A sequence S of ~ p i states defines a time series. The av erage state at time i of states ~ p 1 through ~ p j is defined as: ~ a i = 1 i k X j =1 ~ p j (10) The opp osite state defines an opp osition measure from ~ p i as ~ r i = ~ p i + 2( ~ a i − ~ p i ) (11) and in this w ay an opp osition vector can b e defined: ~ D i = ~ r i − ~ p i . (12) Kno wing that an y displacement from one state ~ p i to another state ~ p j is defined as ~ M i,j = ~ p j − ~ p i (13) it is p ossible to define an opp osition index to quan tify how m uch a protot yp e p j opp oses p i (a displacement in direction of ~ r i ) or emphasis p i (a displacement in − ~ r i direction): W i,j = D ~ M i,j , ~ D i E || ~ D i || 2 (14) A Quantitative Appr o ach to Painting Styles 9 Figure 2. a) The original paintings image. b) A segmented region. c) The extracted curv ature of segmen t. d) The parametric curve k ( t ) with p eaks giv en b y a particular threshold. Ho wev er, the mov ements in such p ainting sp ac e are not restricted to confirmation or refutation of “ideas”. Alternative ideas can exist out of this dualistic displacemen t. This is mo deled as a skewness index which quan tifies how m uc h a prototype p j is inno v ative when compared with p i : s i,j = s | ~ p i − ~ p j | 2 | ~ a i − ~ p i | 2 − [( ~ p i − ~ p j )( ~ a i − ~ p i )] 2 | ~ a i − ~ p i | 2 (15) Another measure arises when considering three consecutiv e states at times i, j and k . Being p i the thesis, p j the antithesis and p k the synthesis, a c ounter-diale ctics index can b e defined b eing d i → k = | h ~ v j − ~ v i , ~ v k i + 1 2 h ~ v i − ~ v j , ~ v i + ~ v j i | | ~ v j − ~ v i | (16) A Quantitative Appr o ach to Painting Styles 10 or, the distance b etw een p k and the middle-line (or middle-h yp erplane for N -dimensional spaces) betw een p i and p j . In other words, a p k state with higher d i → k is far from the syn thesis (low dialectics) and vice-versa. 2.5. F e atur e sele ction T o select the most relev an t features a disp ersion measure of the clusters is applied using scatter matrices [15]. F or all the N pain tings, considering all p ossible com binations of feature pairs F N ,a and F N ,b , the S b (b et w een class) and S w (within class) scatter matrices are calculated with K = 12 classes, one class C i for each pain ter: S w = K X i =1 S i (17) S b = K X i =1 N i ( ~ µ i − ~ M )( ~ µ i − ~ M ) T (18) with N p the num b er of paintings in class C p and the scatter matrix for class C i defined as S i = X i ∈ C i ( ~ f i − ~ µ i )( ~ f i − ~ µ i ) T (19) where ~ f i is an ob ject of the feature matrix F whose rows and columns corresp ond to the pain tings and its features F =  ← f T i →  and ~ µ p and ~ M are the mean feature v ectors for ob jects in class C p and for all the paintings, resp ectively: ~ µ p = 1 N p X i ∈ C p ~ f i (20) ~ M = 1 N N X i =1 ~ f i (21) The trace of within- and b etw een-class ratio can b e used to quantify disp ersion: α = tr( S b S − 1 w ) (22) Large v alues of α reveal larger disp ersion and the features which relate with large v alues of α are selected for the analysis (Section 3.1). 3. Results and discussion 3.1. Best fe atur es By calculating α using Eq. 22 for all p ossible feature pairs F N ,a and F N ,b of the N = 93 features and ordering the results b y α , it is p ossible to select the features whic h are most relev ant to classification: pairs with high α present b etter disp ersion and clustering than A Quantitative Appr o ach to Painting Styles 11 pairs with lo wer v alues. As shown in T able 4 (and Figure 3), features µ of curvatur e p e aks and µ of numb er of se gments hav e the higher α and are selected to opp osition, sk ewness and dialectics analysis — b oth features are sho wn as predominant also in LDA, discussed in next section. It is in teresting to note the nature of selected fea tures: the n umber of segmen ts and curv ature p eaks are the most prominent characteristics for the classification of paintings, even better than texture or image complexit y . Other features presen ting large v alues of α — lik e µ of conv ex-hull area, segmen ts p erimeter and area, and circularit y — are also related with shap e c haracteristics. Both features presented a similar pro jection and clustering prop erties of Figure 3 as sho w ed in Figure A1. T able 4. F eature pairs F N ,a and F N ,b ordered by α . Pairs with higher α presen t b etter disp ersion and clustering. The b est feature pairs µ of curvatur e p e aks and µ of numb er of se gments are selected for analysis and metrics calculation. P air nr. F eature a F eature b α 1 µ of curv ature p eaks µ of n um b er of seg. 42.445 2 µ of n um b er of seg. µ of con v ex-h ull area 37.406 3 µ of segmen ts p erimeter µ of n um b er of seg. 36.703 4 µ of segmen ts area µ of n um b er of seg. 36.214 5 µ of n um b er of segments µ con v ex / original 34.885 6 µ of circularit y (P er . 2 / Area) µ of n um b er of seg. 33.540 7 Energy µ of image ro ws (green) µ of n um b er of seg. 32.954 8 Energy µ of ro ws and columns (green) µ of num b er of seg. 32.954 9 Energy σ of image rows (green) µ of num b er of seg. 32.932 10 Energy σ of rows and columns (green) µ of num b er of seg. 32.906 11 µ of lo cal en trop y (5-size windo w) µ of num b er of seg. 32.898 12 En trop y (Haralick adj. 4) µ of num b er of seg. 32.898 13 En trop y (Haralick adj. 3) µ of num b er of seg. 32.883 14 En trop y (Haralick adj. 1) µ of num b er of seg. 32.874 15 En trop y (Haralick adj. 2) µ of num b er of seg. 32.869 16 Energy µ of image ro ws (r.) µ of n um b er of seg. 32.865 The pro jected p ainting sp ac e considering all the pain tings that are “represen ted” b y ~ p i is presen ted in Figure 3 which rev eals well clustered groups with minor sup erp osition, mainly for mo dern paintings. The time-series S formed by protot yp es ~ p i of each pain ter in to the pro jected space is shown as well. A striking result is the high distance whic h P ollo c k sta ys when compared with the other painters: it is a consequence of the lag n umber of segments presen t in w orks of P ollo c k (the y-axis b eing the pro jection of this feature: µ of se gments numb er ). Therefore, b oth the x-axis ( µ of curvatur e p e aks ) and y-axis are relev an t to separate the baro que and mo dern art mo vemen ts. It is p ossible to note a separation b et w een baro que and mo dern pain ters where the baro que paintings are arranged in an ov erlapping group while the mo dern pain ters are more clustered and separated from eac h other while co vering a widely region of the p ainting sp ac e . This is confirmed by the history of art with modern pain ters being more individualists in their st yles while baro que pain ters are used to share aesthetic c haracteristics in their paintings. The same observ ation arises A Quantitative Appr o ach to Painting Styles 12 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5 µ o f c u r v a t u r e p i k e s 1 0 1 2 3 4 µ o f n u m b e r o f s e g m e n t s 1 Caravaggio 2 Frans Hals 3 Poussin 4: Velazquez 5 Rembrandt 6 Vermeer 7 van Gogh 8 Kandinsky 9 Matisse 10 Picasso 11 Miro 12 Pollock 1: Caravaggio 2: Frans Hals 3: Poussin 4: Velazquez 5: Rembrandt 6: Vermeer 7: van Gogh 8: Kandinsky 9: Matisse 10: Picasso 11: Miro 12: Pollock Figure 3. Pro jected p ainting sp ac e considering the b est pair of features: µ of curvatur e p e aks and µ of numb er of se gments . when follo wing the time-series, the difference b etw een the mov emen ts is clear: while baro que artists tend to presen t a recurring pattern, an abrupt displacemen t separates V an Gogh — the first modern pain ter in the p ainting sp ac e — from the previous, and breaks the cyclic pattern. V an Gogh, although lo cated near the baro que pain ters and in the opp osite extreme of modern painters, represen ts a transition to the modern p erio d and after him the following v ector displacemen ts will contin ue to evolv e un til reac hing its ap ex with P ollo c k. While analyzing the baro que group separately , it is p ossible to observe a tra jectory dra wn b y Cara v aggio and F rans Hals through Poussin which ends with the opp osite A Quantitative Appr o ach to Painting Styles 13 (and bac k forth) mov emen t of V el´ asquez. It can b e attributed to the influence of the “ chiar oscur o ” master into these pain ters, mainly in V el´ asquez who is known to hav e studied the works of Carav aggio [12]. It arises again in the return to the Cara v aggio mo vemen t by V ermeer – some critics affirm [16] that pain ters lik e V ermeer could not ha ve ev en existed without Carav aggio’s influence: V ermeer and Carav aggio clusters are the most s up erimp osed considering all the portraits in the p ainting sp ac e . Both facts are confirmed by the histograms of gra y lev els sho wn in Figure 4. V elazquez a nd V ermeer curv es are more similar to Cara v aggio than the remaining baro que pain ters. 0 50 100 150 200 250 300 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 Caravaggio Velazquez Vermeer 0 50 100 150 200 250 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 Caravaggio Hals Poussin Rembrandt Figure 4. Mean gra y lev els histograms for all the baro que pain ters. V ermeer and V elazquez show more similarity with Carav aggio than other baro que painters. In summary , the baro que group sho ws a strong inter-relationship by comparing with mo dern pain ters where the absence of sup er-imp ositions is remark able. Again, this suggests a strong style-cen tric distinction among artists of the mo dern era while baro que artists shared tec hniques and aesthetic characteristics. This is also confirmed A Quantitative Appr o ach to Painting Styles 14 when comparing the histograms of mo dern paintings in Figure 5: smaller similarities are observ ed b etw een the considered artists, contrasting with baro que pain ters sho wn in Figure 4. 0 50 100 150 200 250 0.000 0.005 0.010 0.015 0.020 Van Gogh Kandinsky Matisse Picasso Miró Pollock Figure 5. Mean gray levels histogram for all the mo dern painters. There is minor similarities b et w een mo dern artists. When considering opposition and skewness, more in teresting results arise, as sho wn in T able 5 and Figure 6. Clearly , the larger v alue for opp osition is attributed to Rem brandt. This is surprising given that the Dutc h master figures as a “coun terp oin t” of baro que ev en b eing part of this art mov emen t [12]. V ermeer also presents strong opp osition and the nature of its pain tings (e.g. domestic interior, use of brigh t colors) could explain this phenomenon. A pattern is sho wn in the b eginning of baro que and mo dern art: an opp osition decrease is present in b oth cases, which is follow ed by an increase in opp osition. Henceforth, a following plateau of high opp osition v alues is observ ed in baro que painters. This plateau happ ens in the transition p erio d b etw een baro que and mo dern art, gradually decreasing while the mo dern artists b egin to take place in history . This decreasing opposition v alues reflects a lo w opposition role b et w een first artists of baro que p erio d and increasing opp osition as long the p erio d is moving in to mo dernism, although skewness v alues remains oscillating and increasing during almost all the time-series. This c haracterizes again a common scene in arts, mostly in mo dernists, each one trying to define his o wn style and preparing to c hange in to a new mo vemen t. In summary , the p ainting sp ac e is mark ed b y constan tly increasing sk ewness, strong opp osition in sp ecific momen ts of its evolution (the transition b et ween baro que and mo dern) and minor opp osition b et ween the artists of the same mov ement. A Quantitative Appr o ach to Painting Styles 15 T able 5. Opp osition and skewness indices for eac h of the tw elve mov es for a painter to the next. P ain ting Mo ve W i,j s i,j Cara v aggio → F rans Hals 1. 0. F rans Hals → Poussin 0.111 0.425 P oussin → V el´ azquez 0.621 0.004 V el´ azquez → Rembrandt 1.258 0.072 Rem brandt → V ermeer 1.152 0.341 V ermeer → V an Gogh 1.158 0.280 V an Gogh → Kandinsky 0.970 0.452 Kandinsky → Matisse 0.089 0.189 Matisse → Picasso 0.117 0.509 Picasso → Mir´ o 0.385 0.325 Mir´ o → P ollo c k 2.376 3.823 C a r a v a g g i o → F r a n s H a l s F r a n s H a l s → P o u s s i n P o u s s i n → V e l a z q u e z V e l a z q u e z → R e m b r a n d t R e m b r a n d t → V e r m e e r V e r m e e r → V a n G o g h V a n G o g h → K a n d i n s k y K a n d i n s k y → M a t i s s e M a t i s s e → P i c a s s o P i c a s s o → M i r o M i r o → P o l l o c k 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 1.00 0.11 0.62 1.26 1.15 1.16 0.97 0.09 0.12 0.39 2.38 0.00 0.43 0.00 0.07 0.34 0.28 0.45 0.19 0.51 0.33 3.82 Opposition Skewness Figure 6. Opp osition W i,j and sk ewness s i,j v alues for the tw o b est features. The counter-dialectics, shown in T able 6 and Figure 7, dra ws a parallel with the opp osition and skewness curv es. It reinforces the already observed facts: pain ters of A Quantitative Appr o ach to Painting Styles 16 the same mo vemen t show initially decreasing follo wed b y increasing coun ter-dialectics reflecting the concordance of members of the same mov ement and their preparation to c hange into the next mo vemen t. The larger coun ter-dialectics happ ens in V an Gogh and Kandinsky: again, the point where baroque ends and mo dern art starts, regarding the pain ters selected for this study . T able 6. Coun ter-dialectics index for each of the ten subsequent mov es among pain ters states for the b est t w o features. P ain ting T riple d i → k Cara v aggio → F rans Hals → Poussin 0.572 F rans Hals → Poussin → V el´ azquez 0.337 P oussin → V el´ azquez → Rembrandt 0.151 V el´ azquez → Rembrandt → V ermeer 0.608 Rem brandt → V ermeer → V an Gogh 1.362 V ermeer → V an Gogh → Kandinsky 1.502 V an Gogh → Kandinsky → Matisse 1.062 Kandinsky → Matisse → Picasso 0.183 Matisse → Picasso → Mir´ o 0.447 Picasso → Mir´ o → P ollo c k 2.616 3.2. Al l the fe atur es Although features F N ,a ( µ of curv ature p eaks) and F N ,b ( µ of n umber of segmen ts) sho wed as an in teresting c hoice for classification, LDA is applied considering all the N = 93 features to test the relev ance of these features and the stabilit y of the results. The LD A metho d [15] pro jected the features in a 2-dimensional space that better separates the pain tings and yields a time-series as done for the tw o most prominen t features. The first t wo comp onents giv e the time-series sho wn in Figure 8. It is p ossible to note, as exp ected, a similarity with results from Subsection 3.1. The sk ewness indices sho w ev en more an ascending curv e along the entire evolution, as presented in T able 7 and Figure 9. The opp osition and dialectics (T able 8 and Figure 10) patterns remain. F or LD A v alidation, the total set of pain tings is split in tw o groups: a training set with 10 random selected pain tings for each artist and a test set with the remaining 10 pain tings for eac h artist, without rep etition. Such a v alidation is p erformed 100 times. The confusion matrix (Figure 11) rev eals the qualit y of predicted output. Diagonal elemen ts represent the mean num b er of samples for which the predicted class is equal to the true class, while off-diagonal elemen ts indicates those ones that are unclassified by LD A. Higher diagonal v alues indicate more correct predictions. As observ ed, the LD A metho d p erformed as expected for the considered set of paintings. The b est classified samples are Pollock paintings which is exp ected giv en the high detachmen t of this cluster observed in the presented pro jections. In general, the confusion matrix reflects facts previously discussed: a similarit y b etw een baro que painters, mainly V elazquez, A Quantitative Appr o ach to Painting Styles 17 C a r a v a g g i o → F r a n s H a l s → P o u s s i n F r a n s H a l s → P o u s s i n → V e l a z q u e z P o u s s i n → V e l a z q u e z → R e m b r a n d t V e l a z q u e z → R e m b r a n d t → V e r m e e r R e m b r a n d t → V e r m e e r → V a n G o g h V e r m e e r → V a n G o g h → K a n d i n s k y V a n G o g h → K a n d i n s k y → M a t i s s e K a n d i n s k y → M a t i s s e → P i c a s s o M a t i s s e → P i c a s s o → M i r o P i c a s s o → M i r o → P o l l o c k 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0.57 0.34 0.15 0.61 1.36 1.50 1.06 0.18 0.45 2.62 Counter-dialectics Figure 7. Counter-dialectics v alues considering the t w o b est features. T able 7. Opposition and skewness indices for each of the tw elv e painters states mo ves P ain ting Mo ve W i,j s i,j Cara v aggio → F rans Hals 1. 0. F rans Hals → Poussin -0.101 0.132 P oussin → V el´ azquez 0.588 0.037 V el´ azquez → Rembrandt 1.526 0.050 Rem brandt → V ermeer 1.101 0.143 V ermeer → V an Gogh 1.153 0.157 V an Gogh → Kandinsky 1.279 0.512 Kandinsky → Matisse 0.179 0.149 Matisse → Picasso -0.201 0.516 Picasso → Mir´ o 0.432 0.163 Mir´ o → P ollo c k 4.031 2.662 Cara v aggio and Rem brandt and a separation b et ween painters b efore and after V an Gogh which defines the frontier b et w een the baro que and mo dern mov emen ts. A Quantitative Appr o ach to Painting Styles 18 80 60 40 20 0 20 40 60 Second Component 200 0 200 400 600 800 First Component 1: Caravaggio 2: Frans Hals 3: Poussin 4: Velazquez 5: Rembrandt 6: Vermeer 7 van Gogh 8 Kandinsky 9 Matisse 10 Picasso 11 Miro 12 Pollock 1: Caravaggio 2: Frans Hals 3: Poussin 4: Velazquez 5: Rembrandt 6: Vermeer 7: van Gogh 8: Kandinsky 9: Matisse 10: Picasso 11: Miro 12: Pollock Figure 8. Time series yielded b y 2-dimensional pro jected “painting space” considering the t wo first comp onents obtained by LD A transformed into the N = 93 feature matrix. 4. Conclusions It is shown that t w o features: a) num b er of curv ature p eaks and b) num b er of segmen ts of an image — b oth related with shap e characteristics — can b e used for the classification of the selected painters with remark able results, ev en when compared with canonical feature measures lik e Haralic k or image complexity . Such relev ance is supp orted by the analysis of a disp ersion index calculated for every pair of features and reinforced b y LD A analysis. The effective c haracterization of selected pain tings b y means of these features allo wed the definition of a “pain ting space”. While represented as states in this A Quantitative Appr o ach to Painting Styles 19 C a r a v a g g i o → F r a n s H a l s F r a n s H a l s → P o u s s i n P o u s s i n → V e l a z q u e z V e l a z q u e z → R e m b r a n d t R e m b r a n d t → V e r m e e r V e r m e e r → V a n G o g h V a n G o g h → K a n d i n s k y K a n d i n s k y → M a t i s s e M a t i s s e → P i c a s s o P i c a s s o → M i r o M i r o → P o l l o c k 1 0 1 2 3 4 5 1.00 -0.10 0.59 1.53 1.10 1.15 1.28 0.18 -0.20 0.43 4.03 0.00 0.13 0.04 0.05 0.14 0.16 0.51 0.15 0.52 0.16 2.66 Opposition Skewness Figure 9. Opp osition and Sk ewness v alues considering the time series for all the features. The same patterns observed when analyzing the b est feature pair remains in this observ ation. T able 8. Coun ter-dialectics index for each of the ten subsequent mov es among pain ters states for the b est t w o comp onen ts of LDA pro jection. P ain ting T riple d i → k Cara v aggio → F rans Hals → Poussin 0.587 F rans Hals → Poussin → V el’azquez 0.317 P oussin → V el’azquez → Rembrandt 0.268 V el’azquez → Rembrandt → V ermeer 0.736 Rem brandt → V ermeer → V an Gogh 1.192 V ermeer → V an Gogh → Kandinsky 2.352 V an Gogh → Kandinsky → Matisse 0.974 Kandinsky → Matisse → Picasso 0.241 Matisse → Picasso → Mir’o 0.704 Picasso → Mir’o → P ollo c k 1.924 A Quantitative Appr o ach to Painting Styles 20 C a r a v a g g i o → F r a n s H a l s → P o u s s i n F r a n s H a l s → P o u s s i n → V e l a z q u e z P o u s s i n → V e l a z q u e z → R e m b r a n d t V e l a z q u e z → R e m b r a n d t → V e r m e e r R e m b r a n d t → V e r m e e r → V a n G o g h V e r m e e r → V a n G o g h → K a n d i n s k y V a n G o g h → K a n d i n s k y → M a t i s s e K a n d i n s k y → M a t i s s e → P i c a s s o M a t i s s e → P i c a s s o → M i r o P i c a s s o → M i r o → P o l l o c k 0.0 0.5 1.0 1.5 2.0 2.5 0.59 0.32 0.27 0.74 1.19 2.35 0.97 0.24 0.70 1.92 Counter-dialectics Figure 10. Counter-dialectics v alues (higher v alues reveals lo wer dialectics) considering all the features. The pattern observ ed in the b est pair pro jection became stronger here: it is p ossible to observe clearly the highest v alue along the mo v ement transition p erio d (V an Gogh and Kandinsky). pro jected space, the baro que pain tings are sho wn as an o v erlapp ed cluster. The mo dern pain tings clusters, in contrast, present minor ov erlapping and are disposed more widely in the pro jection. Those observ ations are compatible with the history of Art: baro que pain ters shared aesthetics while mo dern painters tended to define their o wn styles individually [12]. A time-series — comp osed b y prototype states representing eac h painter c hronologically — allow ed the concepts of opp osition, sk ewness and dialectics to b e approac hed quantitativ ely , as geometric measures. The painting states sho w a decrease in opp osition and dialectics considering the first mem b ers of the same mov ement (baro que or modern) follo wed b y increasing opposition and dialectics un til it reac hes the strong opp osition momen tum b etw een the t wo mo vemen ts. Also, the skewness curve increases during almost en tire time-series. This could reflect a strong influence role of a mo vemen t in its members together with an increasing desire to innov ate, present in A Quantitative Appr o ach to Painting Styles 21 Caravaggio Frans Hals Poussin Velazquez Rembrandt Vermeer Van Gogh Kandinsky Matisse Picasso Miro Pollock Predicted paintings Caravaggio Frans Hals Poussin Velazquez Rembrandt Vermeer Van Gogh Kandinsky Matisse Picasso Miro Pollock True paintings 0 1 2 3 4 5 6 7 8 9 Figure 11. Confusion matrix for LDA. The half of paintings are used as a training set and the other half as test set. The v alidation is performed 100 times. Diagonal elemen ts shows the mean num b er of paintings in the predicted class (a painter) which equals to the true class. eac h artist, stronger in mo dernists. Both opp osition, skewness and dialectics measurements can b e compared with results already obtained for music and philosoph y [1]. Music comp osers seems to b e guided by strong dialectics due to the recognized master-appren tice role. Philosophers mo vemen ts, otherwise, are strong in opp osition. Pain ters, as this study reveals, show increasing skewness and strong opp osition and coun ter-dialectics in sp ecific momen ts of history . While not sufficient to exhaust all the c haracteristics regarding an artist or its work, this metho d suggests a framew ork to the study of arts by means of a feature space and geometrical measures. As a future work, the n um b er of painters could b e increased and a set of painters could b e sp ecifically chosen to analyze influence (e.g. w orks of F rans A Quantitative Appr o ach to Painting Styles 22 Hals sons can be included to verify the influence of their father and master, or pain tings b y Rafael, Poussin and Guido Reni [12] or Carracci can b e compared to confron t the already kno wn similarity of b oth painters). A larger num b er of paintings for eac h artist could be considered to analysis as w ell. The same framew ork can b e applied to other fields of interest lik e Movies or P o etry . Another in teresting use of this framework — b eing currently dev elop ed b y the authors — is a comp onent of a generative art mo del: geometrical measures in the p ainting sp ac e (lik e the already defined dialectics or opp osition and skewness) can guide an evolutionary algorithm, assigning the v alue of measures as the fitness of generated material. This mo del complements a framew ork to the study of creative ev olution in arts. App endix Although the first features pair ( µ of curv ature pik es and µ of n umber of segments) is selected to the analysis, other features with large α v alues can be used as sho wn in Figure A1. References [1] Vilson Vieira, Renato F abbri, Gonzalo T ravieso, Osv aldo N Oliveira Jr, and Luciano da F on toura Costa. A quantitativ e approach to evolution of music and philosophy . Journal of Statistic al Me chanics: The ory and Exp eriment , 2012(08):P08010, 2012. [2] AnaIoana Deac, Jan Lubb e, and Eric Back er. F eature selection for paintings classification b y optimal tree pruning. In Bilge Gunsel, AnilK. Jain, A.Murat T ek alp, and Blent Sankur, editors, Multime dia Content R epr esentation, Classific ation and Se curity , volume 4105 of L e ctur e Notes in Computer Scienc e , pages 354–361. Springer Berlin Heidelb erg, 2006. [3] Oguz Icoglu, Bilge Gunsel, and Sanem Sariel. Classification and indexing of paintings based on art mo v emen ts. In Pr o c. of EUSIPCO , pages 749–752, 2004. [4] M. Sp ehr, C. W allrav en, and R. W. Fleming. Image statistics for clustering paintings according to their visual app earance. In Pr o c e e dings of the Fifth Eur o gr aphics c onfer enc e on Computational A esthetics in Gr aphics, Visualization and Imaging , Computational Aesthetics’09, pages 57–64, Aire-la-Ville, Switzerland, Switzerland, 2009. Eurographics Asso ciation. [5] C.R. Johnson, E. Hendriks, I.J. Berezhnoy , E. Brevdo, S.M. Hughes, I. Daub echies, Jia Li, E. Postma, and J.Z. W ang. Image pro cessing for artist identification. Signal Pr o c essing Magazine, IEEE , 25(4):37–48, 2008. [6] Lev Manovic h. Style space: Ho w to compare image sets and follow their ev olution (draft text). http://lab.softwarestudies.com/2011/08/style- space- how- to- compare- image- sets. html , August 2011. [7] Lev Manovic h. Mondrian vs rothk o: footprints and ev olution in style space. http://lab. softwarestudies.com/2011/06/mondrian- vs- rothko- footprints- and.html , June 2011. [8] Lev Manovic h. Arthistory .viz — visualizing mo dernism. http://lab.softwarestudies.com/ 2008/07/arthistoryviz- mining- 200000- images- of.html , Nov ember 2008. [9] Juan Romero, P enousal Machado, Adrian Carballal, and Antonino Santos. Using complexity estimates in aesthetic image classification. Journal of Mathematics and the Arts , 6(2-3):125– 136, 2012. [10] J. Zujovic, L. Gandy , S. F riedman, B. P ardo, and T.N. Pappas. Classifying pain tings by artistic genre: An analysis of features and classifiers. In Multime dia Signal Pr o c essing, 2009. MMSP ’09. IEEE International Workshop on , pages 1–5, 2009. A Quantitative Appr o ach to Painting Styles 23 p a i r 1 : α = 4 2 . 4 4 5 p a i r 2 : α = 3 7 . 4 0 6 p a i r 3 : α = 3 6 . 7 0 3 p a i r 4 : α = 3 6 . 2 1 4 p a i r 5 : α = 3 4 . 8 5 5 p a i r 6 : α = 3 3 . 5 4 0 p a i r 7 : α = 3 2 . 9 5 4 p a i r 8 : α = 3 2 . 9 5 4 p a i r 9 : α = 3 2 . 9 3 2 p a i r 1 0 : α = 3 2 . 9 0 6 p a i r 1 1 : α = 3 2 . 8 9 8 p a i r 1 2 : α = 3 2 . 8 9 8 p a i r 1 3 : α = 3 2 . 8 8 3 p a i r 1 4 : α = 3 2 . 8 7 4 p a i r 1 5 : α = 3 2 . 8 6 9 p a i r 1 6 : α = 3 2 . 8 6 5 Figure A1. Scatter plots for each feature pair i listed in T able 4 with large v alues of α . The first pro jection (pair 1) was used for the analysis, how ev er other pro jections (pairs 2 . . . 16) can be used. [11] H.L. Williams. He gel, Her aclitus, and Marx’s Diale ctic . St. Martin’s Press, 1989. [12] E.H. Gombric h. The story of art . STOR Y OF AR T. Phaidon Press, Ltd., 1995. [13] Radhakrishna Achan ta, Appu Sha ji, Kevin Smith, Aurlien Lucchi, Pascal F ua, and Sabine Ssstrunk. SLIC Sup erpixels Compared to State-of-the-art Sup erpixel Methods. IEEE Tr ansactions on Pattern Analysis and Machine Intel ligenc e , 34(11):2274 – 2282, 2012. A previous v ersion of this article was published as a EPFL T echnical Rep ort in 2010: h ttp://infoscience.epfl.ch/record/149300. Supplemen tary material can b e found at: h ttp://ivrg.epfl.c h/research/superpixels. [14] Rob ert M Haralic k, Karthikey an Shanmugam, and Its’ Hak Dinstein. T extural features for image classification. Systems, Man and Cyb ernetics, IEEE T r ansactions on , (6):610–621, 1973. [15] Luciano da F ontoura Da Costa and Rob erto Marcondes Cesar, Jr. Shap e Analysis and A Quantitative Appr o ach to Painting Styles 24 Classific ation: The ory and Pr actic e . CR C Press, Inc., Bo ca Raton, FL, USA, 1st edition, 2000. [16] G. Lam b ert and G. N´ eret. Car avaggio. Ediz. te desc a: . Basic Art Series. T aschen Deutschland Gm bH, 2000.