Skip to content

Latest commit

 

History

History
116 lines (94 loc) · 5.13 KB

readme.md

File metadata and controls

116 lines (94 loc) · 5.13 KB
Raw

Dense Intrinsic Appearance Flow for Human Pose Transfer

This is a pytorch implementation of the CVPR 2019 paper Dense Intrinsic Appearance Flow for Human Pose Transfer.

fig_intro

Requirements

  • python 2.7
  • pytorch (0.4.0)
  • numpy
  • opencv
  • scikit-image
  • tqdm
  • imageio

Install dependencies:

pip install -r requirements.txt

Resources

Datasets

Download and unzip preprocessed datasets with the following scripts.

bash scripts/download_deepfashion.sh
bash scripts/download_market1501.sh

Or you can manually download them from the following links:

Pretrained Models

Download pretrained models with the following scripts.

bash scripts/download_models.sh

Pretrained models below will be downloaded into the folder ./checkpoints. You can manually donwload them from here.

Deepfashion Market-1501 Others
  • PoseTransfer_0.1 (w/o. dual encoder)
  • PoseTransfer_0.2 (w/o. flow)
  • PoseTransfer_0.3 (w/o. vis)
  • PoseTransfer_0.4 (w/o. pxiel warping)
  • PoseTransfer_0.5 (full)
  • PoseTransfer_m0.1 (w/o. dual encoder)
  • PoseTransfer_m0.2 (w/o. flow)
  • PoseTransfer_m0.3 (w/o. vis)
  • PoseTransfer_m0.4 (w/o. pxiel warping)
  • PoseTransfer_m0.5 (full)
  • Fasion_Inception(compute FashionIS)
  • Fasion_Attr(compute AttrRec-k)

Testing

DeepFashion

  1. Run scripts/test_pose_transfer.py to generate images and compute SSIM score.
python scripts/test_pose_transfer.py --gpu_ids 0 --id PoseTransfer_0.5 --which_epoch best --save_output
  1. Compute inception score with the following script. (Note that this script is derived from improved-gan and needs Tensorflow)
# python scripts/inception_score.py image_dir gpu_ids
python scripts/inception_score.py checkpoints/PoseTransfer_0.5/output/ 0
  1. Compute fashionIS and AttrRec-k with the following scripts.
# FashionIS
python scripts/fashion_inception_score.py --test_dir checkpoints/PoseTransfer_0.5/output/

# AttrRec-k
python scripts/fashion_attribute_score.py --test_dir checkpoints/PoseTransfer_0.5/output/

Market-1501

  1. Run scripts/test_pose_transfer.py to generate images and compute SSIM/masked-SSIM score.
python scripts/test_pose_transfer.py --gpu_ids 0 --id PoseTransfer_m0.5 --which_epoch best --save_output --masked
  1. Compute inception score or masked inception score with following scripts.
# IS
python scripts/inception_score.py checkpoints/PoseTransfer_m0.5/output/ 0

# masked-IS (only for market-1501)
python scripts/masked_inception_score.py checkpoints/PoseTransfer_m0.5/output/ 0

Training

DeepFashion

  1. Train flow regression module. (See all options in ./options/flow_regression_options.py)
python scripts/train_flow_regression_module.py --id id_flow --gpu_ids 0 --which_model unet --dataset_name deepfashion

You can alternativelly set --which_model unet_v2 to use a improved version of network architecture with fewer parameters (only tested on Market-1501).

  1. Train human pose transfer models. Set --pretrained_flow_id and --pretrained_flow_epoch to load the flow regression module. (See all options in ./options/pose_transfer_options.py)
# w/o. dual encoder
python scripts/train_pose_transfer_model.py --id id_pose_1 --gpu_ids 1 --dataset_name deepfashion --which_model_G unet

# w/o. flow
python scripts/train_pose_transfer_model.py --id id_pose_2 --gpu_ids 2 --dataset_name deepfashion --which_model_G dual_unet --G_feat_warp 0

# w/o. visibility
python scripts/train_pose_transfer_model.py --id id_pose_3 --gpu_ids 3 --dataset_name deepfashion --which_model_G dual_unet --G_feat_warp 1 --G_vis_mode none

# w/o. pixel warping
python scripts/train_pose_transfer_model.py --id id_pose_4 --gpu_ids 4 --dataset_name deepfashion --which_model_G dual_unet --G_feat_warp 1 --G_vis_mode residual

# full (need a pretrained pose transfer model without pixel warping)
python scripts/train_pose_transfer_model.py --id id_pose_5 --gpu_ids 5 --dataset_name deepfashion --G_pix_warp 1 --which_model_G dual_unet --pretrained_G_id id_pose_4 --pretrained_G_epoch 8

Market-1501

Set --dataset_name market to train models on Market-1501 dataset. Data related parameters will be automatically adjusted (see .auto_set() in ./options/flow_regression_options.py and ./options/pose_transfer_options.py for details).

Citation

@inproceedings{li2019dense,
  author = {Li, Yining and Huang, Chen and Loy, Chen Change},
  title = {Dense Intrinsic Appearance Flow for Human Pose Transfer},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
  year = {2019}}