Multilinear Biased Discriminant Analysis: A Novel Method for Facial Action Unit Representation

Abstract In this paper a novel efficient method for representation of facial action units by encoding an image sequence as a fourth-order tensor is presented. The multilinear tensor-based extension of the biased discriminant analysis (BDA) algorithm, called multilinear biased discriminant analysis (MBDA), is first proposed. Then, we apply the MBDA and two-dimensional BDA (2DBDA) algorithms, as the dimensionality reduction techniques, to Gabor representations and the geometric features of the input image sequence respectively. The proposed scheme can deal with the asymmetry between positive and negative samples as well as curse of dimensionality dilemma. Extensive experiments on Cohn-Kanade database show the superiority of the proposed method for representation of the subtle changes and the temporal information involved in formation of the facial expressions. As an accurate tool, this representation can be applied to many areas such as recognition of spontaneous and deliberate facial expressions, multi modal/media human computer interaction and lie detection efforts. 1 Introduction Human face-to-face communication is a standard of perfection for developing a natural, robust, effective and flexible multi modal/media human-computer interface due to multimodality and multiplicity of its communication channels. In this type of communication, the facial expressions constitute the main modality [1]. In this regard, automatic facial expression analysis can use the facial signals as a new modality and causes the interaction between human and computer more robust and flexible. Moreover, it can be used in other areas such as lie detection, neurology and clinical psychology.
Facial expression analysis includes both measurement of facial motion (e.g. mouth stretch or outer brow raiser) and recognition of expression (e.g. surprise or anger). Real-time fully automatic facial expression analysis is a challenging complex topic in computer vision due to pose variations, illumination variations, different age, gender, ethnicity, facial hair, occlusion, head motions and lower intensity of expressions. Two survey papers summarized the work of facial expression analysis [2, 3]. Regardless of the face detection stage, a typical automatic facial expression analysis consists of facial expression data extraction and facial expression classification stages. Facial feature processing may happen either holistically, where the face is processed as a whole, or locally. Holistic feature extraction methods are good at determining prevalent facial expressions, whereas local methods are able to detect subtle changes in small areas.
There are mainly two methods for facial data extraction: geometric feature-based methods and appearance-based method. The geometric facial features present the shape and locations of facial components. With appearance-based methods, image filters, e.g. Gabor wavelets, are applied to either the whole face or specific regions in a face image to extract a feature vector [4]. The sequence-based recognition method uses the temporal information of the sequences to recognize the expressions of one or more frames. To use the temporal information, the techniques such as hidden Markov models (HMMs), recurrent neural networks and rule-based classifier were applied.
The main goal of this paper is developing an accurate method for representation of facial action units (AUs). Our method has the following characteristics: 1) it can deal with the asymmetry between positive and negative image sequences as well as curse of dimensionality dilemma, 2) multiple interrelated subspaces can cooperate to discriminate positive and negative image sequences in the new subspace, 3) increasing the recognition rate by using both geometric and appearance features, and 4) it is robust to illumination changes and can represent subtle changes in facial muscles as well as temporal information involved in formation of the expressions.
The rest of the paper has been organized as follows:
In section 2, we review the related works. In section 3, we first describe the proposed approach for representation of facial action units using fourth-order tensors. Then, multilinear biased discriminant analysis (MBDA) algorithm, for reducing the dimensionality of

2 the tensor in all directions, is proposed. Section 4 reports our experimental results, and section 5 presents conclusions and a discussion. 2 Related Works 2.1 Facial Action Coding System The facial action coding system (FACS) is a system developed by Ekman and Friesen to detect subtle changes in facial features. The FACS is co

…(Full text truncated)…