Facial Action Unit Detection with Multilayer Fused Multi-Task and Multi-Label Deep Learning Network

• Jun He,
• Dongliang Li,
• Sun Bo,
• Lejun Yu,

Vol. 13, No. 11, November 30, 2019
10.3837/tiis.2019.11.015, Download Paper (Free):

• facial action unit
• multi-task learning
• multi-label learning
• multilayer fusion
• Deep Learning

Abstract

Facial action units (AUs) have recently drawn increased attention because they can be used to recognize facial expressions. A variety of methods have been designed for frontal-view AU detection, but few have been able to handle multi-view face images. In this paper we propose a method for multi-view facial AU detection using a fused multilayer, multi-task, and multi-label deep learning network. The network can complete two tasks: AU detection and facial view detection. AU detection is a multi-label problem and facial view detection is a single-label problem. A residual network and multilayer fusion are applied to obtain more representative features. Our method is effective and performs well. The F1 score on FERA 2017 is 13.1% higher than the baseline. The facial view recognition accuracy is 0.991. This shows that our multi-task, multi-label model could achieve good performance on the two tasks.

Statistics

Cite this article