Image Pixel Fusion for Human Face Recognition

In this paper we present a technique for fusion of optical and thermal face images based on image pixel fusion approach. Out of several factors, which affect face recognition performance in case of visual images, illumination changes are a significant factor that needs to be addressed. Thermal images are better in handling illumination conditions but not very consistent in capturing texture details of the faces. Other factors like sunglasses, beard, moustache etc also play active role in adding complicacies to the recognition process. Fusion of thermal and visual images is a solution to overcome the drawbacks present in the individual thermal and visual face images. Here fused images are projected into an eigenspace and the projected images are classified using a radial basis function (RBF) neural network and also by a multi-layer perceptron (MLP). In the experiments Object Tracking and Classification Beyond Visible Spectrum (OTCBVS) database benchmark for thermal and visual face images have been used. Comparison of experimental results show that the proposed approach performs significantly well in recognizing face images with a success rate of 96% and 95.07% for RBF Neural Network and MLP respectively.

💡 Research Summary

This paper, titled “Image Pixel Fusion for Human Face Recognition,” presents a novel and robust approach to face recognition by addressing long-standing challenges such as illumination variations, facial disguises (e.g., glasses, beards), and expression changes. The core innovation lies in fusing information from two complementary imaging modalities: visual (optical) images and thermal (infrared) images.

The authors begin by outlining the limitations of single-modality systems. Visual spectrum recognition, while rich in texture details, is highly sensitive to lighting conditions, where changes in illumination can cause greater intra-person variation than inter-person differences. Thermal imaging, which captures the heat pattern emitted from facial vasculature, is largely invariant to visible light and more resistant to disguises but lacks consistent textural detail. To overcome these individual drawbacks, the paper proposes a pixel-level fusion strategy.

The system pipeline consists of three main stages. First, a registered pair of visual and thermal images from the same subject undergoes pixel-wise weighted fusion. The fusion formula is F(x,y) = aV(x,y) + bT(x,y), where V is the visual image, T is the thermal image, and the weights are empirically set to a=0.70 and b=0.30. This creates a new, more informative “Common Relevant Operating Picture (CROP).”

Second, dimensionality reduction and feature extraction are performed using the Eigenface method (Principal Component Analysis). A set of eigenfaces is constructed from a training set of fused images, defining a lower-dimensional “face space.” Both training and testing fused images are then projected onto this eigenspace to obtain compact feature vectors (eigenface coefficients).

Third, these feature vectors are classified. The paper employs and compares two neural network classifiers: a Radial Basis Function (RBF) network and a Multi-Layer Perceptron (MLP). A significant technical discussion is dedicated to the RBF network’s design. The authors argue against using standard unsupervised clustering (like k-means) for initializing the RBF centers, as it ignores class label information and can lead to poor initial clusters. Instead, they advocate for a supervised clustering procedure that considers the target class membership, emphasizing that the Gaussian width (spread) of each RBF unit is as crucial as the center location for defining class boundaries.

Experiments were conducted using the Object Tracking and Classification Beyond Visible Spectrum (OTCBVS) database benchmark. The results demonstrated the superior performance of the proposed fusion approach. The RBF neural network classifier achieved a recognition success rate of 96%, while the MLP classifier achieved 95.07%. These high rates confirm the synergistic effect of combining visual and thermal information, where the fused image retains the texture details from the visual domain and the illumination-invariant anatomical features from the thermal domain.

In conclusion, this work provides a comprehensive framework for multi-modal face recognition. It effectively combines pixel-level sensor fusion, subspace projection for efficient representation, and advanced neural network classification. The study not only presents a method with high accuracy but also offers valuable insights into the design of RBF networks for pattern recognition tasks, highlighting the importance of supervised initialization in achieving optimal performance.