A Study on Fusion Strategies of Facial Landmark-Based Heatmap for Facial Expression Recognition

• Hong-Quan Do,
• Hoang Van Thanh,
• Tu Minh Phuong,

Vol. 19, No. 10, October 31, 2025
10.3837/tiis.2025.10.015, Download Paper (Free):

• data fusion
• facial expression recognition
• facial landmark
• heatmap
• Image Classification

Abstract

Facial Expression Recognition (FER) represents a crucial topic in computer vision and affective computing, focusing on automatically identifying human emotions through facial images. While recent developments in FER have been predominantly driven by deep learning architectures such as CNN-based and Transformer-based networks with promising results, these approaches primarily extract features directly from raw facial images. Our study reveals that incorporating facial landmark information in a meaningful way leads to improved performance. Specifically, using heatmaps generated from landmarks produces better results than using raw landmark coordinates. Secondly, the fusion approach significantly impacts performance, with early fusion yielding the best results. Finally, selective landmark points contribute more effectively to expression recognition than utilizing the complete set of facial landmarks. Through systematic experiments, we furthermore identify the optimal standard deviation value for Gaussian heatmap generation.

Statistics

Cite this article