Facial Analysis in the Era of Mobile Devices and Face Masks

Summary

Occlusion may appear in unconstrained environments, but it is now an issue even in controlled setups due to mandatory masks. Solutions must be also capable to operate on devices with hardware restrictions, a necessity if they are to be employed on devices such as smartphones or assistive robots in home or healthcare environments.

One project goal is to provide reliable methods to detect the face. Impressive performance is shown by deep learning solutions, but they use heavy Convolutional Neural Networks (CNN) of hundreds of megabytes, infeasible in mobiles or robots. Also, most are trained to detect the entire face, and not specifically to cope with occlusion. The researchers will use complex symmetry filters as attention mechanism to facilitate detection, coupled with CNN's with complex coefficients, given the success of such filters as a stand-alone method to detect face landmarks in controlled conditions.

Another goal is the estimation of soft biometrics indicators (gender, age, ethnicity). These indicators are easier to extract in unconstrained scenarios and can complement a non-conclusive result of a hard modality (iris or face). They have other applications as well, such as customized advertising, enhanced HCI, age-dependent access, location of specific individuals in video streams, or child pornography detection. However, the use of ocular images for such task and with light CNNs are unexplored avenues.