Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network

Paper: Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network

by Xin Wang, Bo Wu, Yueqi Zhong. Published at ACM MM 2019 in Nice, France.

[Paper] [Arxiv] [Demo]

Contents of this repository

• mcn: Main program source code
• data: Polyvore-T datasets based on Polyvore.
• baselines: Compared baselines in our experiment
• exp: Experiment details, scripts and results etc.

Requirements

Ubuntu 16.04, NVIDIA GTX 1080Ti (for batch size 16), python >= 3.5.2

torch==1.0.1
torchvision==0.2.1
networkx==2.4
opencv-python==4.2.0.32
matplotlib==2.2.2
scikit-learn==0.21.2

Usage

1. Download the original Polyvore dataset, then unzip the file and put the image directory into data folders (or you can create a soft link for it).

2. Train

   cd mcn
   python train.py

3. Evaluate

   python evaluate.py
   

4. Visualize outfit diagnosis

   cd exp
   python diagnosis.py
   

5. Automatically revise outfit

   python revision.py
   

Prediction Performance

Pretrained model weights can be found in the links. The train, validation and test split is provided in data.

	AUC	FITB
Pooling	88.35	57.28
Concatenation	83.40	52.91
Self-attention	79.65	48.60
BiLSTM	74.82	46.02
CSN	84.90	57.06
Ours	91.90	64.35

Demo

A demo application is in the app directioy. You can run it locally by go to app directory then use command python main.py.

More guide can be found in here.

Cite

Please cite our paper if you use or refer this code:

@inproceedings{wang2019diagnosis,
  title={Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network},
  author={Xin Wang, Bo Wu and Yueqi Zhong},
  booktitle={ACM International Conference on Multimedia},
  year={2019}
}