Automatic facial recognition has a wide range of applications in the commercial, military, and government sectors, spanning from tagging people in social networking websites to surveillance for homeland security. To date, face recognition research has predominantly focused on the visible spectrum, addressing challenges such as illumination variations, pose, and image resolution. However, for surveillance during nighttime the lack of illumination prevents cameras operating in the visible-light spectrum from being used discreetly and effectively.
Thermal imaging measures radiation in the mid-wave infrared (MWIR) and long-wave infrared (LWIR) spectra, which is naturally emitted by living tissue, and therefore is a highly practical imaging modality for nighttime operation. However, as most databases and watch lists only contain facial imagery in the visible spectrum, it is difficult to match an unknown thermal image to a set of known visible images. This is referred to as cross-modal face recognition.
The Army’s solution to the above problem is a cross-modal face matching system using polarimetric thermal image data. Polarimetric imaging in the thermal spectrum is sensitive to changes in surface texture and geometry. The polarization-state of radiation emission provides geometric and texture information about the surface of the imaged face. For cross-modal recognition, the combination of polarimetric face features with conventional thermal face features provides a stronger correlation with the visible light feature representation and leads to better matching results than conventional thermal imaging alone.
The novel method comprises the input of many polarimetric images of a face acquired by a thermal imaging camera, extraction of features from each of the images to generate feature vectors for each of the images, creating a composition of the feature vectors for each of the images together to form one composite image, and cross-matching the composite with other feature vectors in order to determine a match.
- The composite feature vector set can be matched with cross-modal data, (thermal images, visible images) to produce a very accurate matching result
- The thermal polarimetric camera and the facial recognition apparatus do not require a light source to illuminate the face
- A pre-processing module may perform functions such as increasing the signal-to-noise ratio by averaging across several frames captured by the polarimetric thermal camera, removing speckle noise, averaging pixel values, registering images to canonical coordinates, and filtering the images to accentuate edge features
- Every pixel in the processed image data contributes to a feature vector that contains at least a direction and strength of an edge/feature for that pixel
- US patent 9,830,506 available for license
- Potential for collaboration with Army researchers