Exploring the Impact of Facial Features on Apparent Personality Traits Detection Using Deep Learning Techniques

Abstract

Apparent personality detection has emerged as a prominent research area within deep learning. While numerous deep learning solutions have been developed to predict personality accurately, the lack of transparency in how these models derive predictions based on facial features undermines trust in their results. This study focuses on identifying and differentiating facial features that contribute to the Big-Five personality traits, addressing transparency in model predictions. To conduct our experiments, we utilised the ChaLearn First Impressions V2 dataset, with background removed frames ensuring models focused more on human features than background in the learning process. We began by developing Convolutional Neural Networks architectures using pre-trained VGGFace and VGG19 models. Subsequently, we employed the Grad-CAM and Guided Grad-CAM model explainable AI techniques on the test and validation datasets, utilising the trained models. Furthermore, we employed the "SelectKBest" feature selection method to analyse the outcomes of the interpretability techniques. VGG19 achieved higher accuracy (90%) compared to VGGFace (89%). Our investigation reveals that personality prediction extends beyond facial features, with XAI techniques emphasizing non-facial aspects such as background information. Statistical analysis across deep learning architectures shows no significant correlation between features identified by XAI techniques by giving different F1-scores. Despite VGG19's superior accuracy, it exhibits a stronger inclination towards non-facial data, while VGGFace prioritizes facial features, highlighting the nuanced nature of personality prediction and suggesting avenues for further research.