Skip to content

Latest commit

 

History

History
125 lines (102 loc) · 6.56 KB

README.md

File metadata and controls

125 lines (102 loc) · 6.56 KB

AUNCE

  • Paper title : Learning Contrastive Feature Representations for Facial Action Unit Detection

Requirements

  • Python 3

  • torch >= 1.4.0

  • torchvision

  • pillow

  • numpy

  • tqdm

  • timm

  • easydict

  • pyyaml == 5.4.1

  • Check the required python packages in requirements.txt.

pip install -r requirements.txt

Data and Data Prepareing Tools

The Datasets we used:

We provide tools for prepareing data in tool/. After Downloading raw data files, you can use these tools to process them, aligning with our protocals.

Training with ImageNet pre-trained models

Make sure that you download the ImageNet pre-trained models to checkpoints/ (or you alter the checkpoint path setting in model/swin_transformer.py)

The download links of pre-trained models are in checkpoints/checkpoints.txt

Training

  • to train our approach on BP4D Dataset, run:
python train_graph_au.py --dataset "BP4D" --exp_name "Graphau_bp4d_swin_nce_step_1" --fold 1 --gpu_ids '0' --info_nce 'enhance'
  • to train our approach on DISFA Dataset, run:
python train_graph_au.py --dataset "DISFA" --exp_name "Graphau_disfa_swin_nce_step_1" --fold 1 --gpu_ids '0' --info_nce 'enhance'

Testing

We adhere to the established linear evaluation protocol, as commonly employed in previous studies CPC and Simclr. Please refer to this link.

BP4D_Sequence_split

Fold1: 'F001','M007','F018','F008','F002','M004','F010','F009','M012','M001','F016','M014','F023','M008'

Fold2: 'M011','F003','M010','M002','F005','F022','M018','M017','F013','M016','F020','F011','M013','M005'

Fold3: 'F007','F015','F006','F019','M006','M009','F012','M003','F004','F021','F017','M015','F014'

DISFA_Sequence_split

Fold1: 'SN002','SN010','SN001','SN026','SN027','SN032','SN030','SN009','SN016'

Fold2: 'SN013','SN018','SN011','SN028','SN012','SN006','SN031','SN021','SN024'

Fold3: 'SN003','SN029','SN023','SN025','SN008','SN005','SN007','SN017','SN004'

Main Results

BP4D

Method Source AU1 AU2 AU4 AU6 AU7 AU10 AU12 AU14 AU15 AU17 AU23 AU24 Avg.
SimCLR 2020 ICML 38.0 36.4 37.2 66.6 64.7 76.2 76.2 51.1 29.8 56.1 27.5 37.7 49.8
MoCo 2020 CVPR 30.8 41.3 42.1 70.2 70.4 78.7 82.5 53.3 25.2 59.1 31.5 34.3 51.6
EAC-Net 2018 TPAMI 39.0 35.2 48.6 76.1 72.9 81.9 86.2 58.8 37.5 59.1 35.9 35.8 55.9
ROI 2017 CVPR 36.2 31.6 43.4 77.1 73.7 85.0 87.0 62.6 45.7 58.0 38.3 37.4 56.4
ARL 2019 TAC 45.8 39.8 55.1 75.7 77.2 82.3 86.6 58.8 47.6 62.1 47.4 55.4 61.1
EmoCo 2021 FG 50.2 44.7 53.9 74.8 76.6 83.7 87.9 61.7 47.6 59.8 46.9 54.6 61.9
MAL 2023 TAC 47.9 49.5 52.1 77.6 77.8 82.8 88.3 66.4 49.7 59.7 45.2 48.5 62.2
CLP 2023 TIP 47.7 50.9 49.5 75.8 78.7 80.2 84.1 67.1 52.0 62.7 45.7 54.8 62.4
JÂA-Net 2021 IJCV 53.8 47.8 58.2 78.5 75.8 82.7 88.2 63.7 43.3 61.8 45.6 49.9 62.4
MMA-Net 2023 PRL 52.5 50.9 58.3 76.3 75.7 83.8 87.9 63.8 48.7 61.7 46.5 54.4 63.4
GeoConv 2022 PR 48.4 44.2 59.9 78.4 75.6 83.6 86.7 65.0 53.0 64.7 49.5 54.1 63.6
AAR 2023 TIP 53.2 47.7 56.7 75.9 79.1 82.9 88.6 60.5 51.5 61.9 51.0 56.8 63.8
SEV-Net 2021 CVPR 58.2 50.4 58.3 81.9 73.9 87.7 87.5 61.6 52.6 62.2 44.6 47.6 63.9
KSRL 2022 CVPR 53.3 47.4 56.2 79.4 80.7 85.1 89.0 67.4 55.9 61.9 48.5 49.0 64.5
AC2D 2024 IJCV 54.2 54.7 56.5 77.0 76.2 84.0 89.0 63.6 54.8 63.6 46.5 54.8 64.6
MEGraph 2022 IJCAI 52.7 44.3 60.9 79.9 80.1 85.3 89.2 69.4 55.4 64.4 49.8 55.1 65.5
SACL 2024 TAC 57.8 48.8 59.4 79.1 78.8 84.0 88.2 65.2 56.1 63.8 50.8 55.2 65.6
CLEF 2023 ICCV 55.8 46.8 63.3 79.5 77.6 83.6 87.8 67.3 55.2 63.5 53.0 57.8 65.9
AUNCE(Ours) - 53.6 49.8 61.6 78.4 78.8 84.7 89.6 67.4 55.1 65.4 50.9 58.0 66.1

DISFA

Method Source AU1 AU2 AU4 AU6 AU9 AU12 AU25 AU26 Avg.
SIMCLR 2020 ICML 21.2 23.3 47.5 42.4 35.5 66.8 81.5 52.7 46.4
MoCo 2020 CVPR 22.7 18.2 45.9 45.4 34.1 72.9 83.4 54.5 47.1
EAC-Net 2018 TPAMI 41.5 26.4 66.4 50.7 80.5 89.3 88.9 15.6 48.5
ROI 2017 CVPR 41.5 26.4 66.4 50.7 80.5 89.3 88.9 15.6 48.5
ARL 2019 TAC 43.9 42.1 63.6 41.8 40.0 76.2 95.2 66.8 58.7
CLP 2023 TIP 42.4 38.7 63.5 59.7 38.9 73.0 85.0 58.1 57.4
MAL 2023 TAC 43.8 39.3 68.9 47.4 48.6 72.7 90.6 52.6 58.0
EmoCo 2021 FG 42.7 41.0 66.3 45.1 50.9 75.5 88.9 58.6 58.6
SEV-Net 2021 CVPR 55.3 53.1 61.5 53.6 38.2 71.6 95.7 41.5 58.8
GeoConv 2022 PR 65.5 65.8 67.2 48.6 51.4 72.6 80.9 44.9 62.1
MEGraph 2022 IJCAI 52.5 45.7 76.1 51.8 46.5 76.1 92.9 57.6 62.4
JÂA-Net 2021 IJCV 62.4 60.7 67.1 41.1 45.1 73.5 90.9 67.4 63.5
AAR 2023 TIP 62.4 53.6 71.5 39.0 48.8 76.1 91.3 70.6 64.2
KSRL 2022 CVPR 60.4 59.2 67.5 52.7 51.5 76.1 91.3 57.7 64.5
CLEF 2023 ICCV 64.3 61.8 68.4 49.0 55.2 72.9 89.9 57.0 64.8
AC2D 2024 IJCV 57.8 59.2 70.1 50.1 54.4 75.1 90.3 66.2 65.4
SACL 2024 TAC 62.0 65.7 74.5 53.2 43.1 76.9 95.6 53.1 65.5
MMA-Net 2023 PRL 63.8 54.8 73.6 39.2 61.5 73.1 92.3 70.5 66.0
AUNCE(Ours) - 61.8 58.9 74.9 49.7 56.2 73.5 92.1 64.2 66.4

Pretrained models

The trained models can be downloaded here.

Citation

Our paper will come soon.