MEAN architecture

Characteristics:

• Shallow network
• Multi-stream
• Multi-output with two task-specific networks
• Capable of both micro-expression spotting and recognition

Results

Take note that analysis is equivalent to spot-then-recognize and STRS is our proposed evaluation metric.
Please refer to the paper for more experiment results.
Here is the detailed result for micro-expression spotting (S) and analysis (A):

How to run the code

Step 1) Download the micro-expression datasets for experiment, we suggest the files to be structured as follows:

├─MEAN_Weights
├─Utils
├─define_model.py
├─face_crop.py
├─feature_extraction.py
├─load_excel.py
├─load_images.py
├─main.py
├─prepare_training.py
├─train_evaluate.py
├─training_utils.py
├─requirements.txt
├─CASME_sq

├─code_final.xlsx
├─rawpic

├─CASME2

├─CASME2

├─CASME2-RAW
└─CASME2_label_Ver_2.xls

├─SAMM

└─SAMM_20181215_Micro_FACS_Codes_v2.xlsx

├─SAMMLV

├─SAMM_longvideos
└─SAMM_LongVideos_V2_Release.xlsx

└─SMIC

├─SMIC-E_raw image
├─HS_long

├─SMIC-HS-E

├─HS
└─SMIC-HS-E_annotation.xlsx

├─NIR_long

└─SMIC-NIR-E

├─NIR
└─SMIC-NIR-E_annotation.xlsx

└─VIS_long

└─SMIC-VIS-E

├─VIS
└─SMIC-VIS-E_annotation.xlsx

Step 2) Installation of packages using pip

pip install -r requirements.txt

Step 3) MEAN Training and Evaluation

python main.py

  Note for parameter settings

   --dataset_name (CASME_sq/SAMMLV/CASME2/SMIC_HS/SMIC_VIS/SMIC_NIR)
   --train (True/False)

Additional Notes

If you find this work useful, please cite the paper:

@article{liong2023spot,
title={Spot-then-Recognize: A Micro-Expression Analysis Network for Seamless Evaluation of Long Videos},
author={Liong, Gen-Bing and See, John and Chan, Chee-Seng},
journal={Signal Processing: Image Communication},
volume={110},
pages={116875},
year={2023},
publisher={Elsevier}
}

Please email me at [email protected] if you have any inquiries or issues.