Skip to content

Latest commit

 

History

History
109 lines (89 loc) · 3.76 KB

README.md

File metadata and controls

109 lines (89 loc) · 3.76 KB
Raw

Compositional GAN in PyTorch

This is the implementation of the Compositional GAN: Learning Image-Conditional Binary Composition. The code was written by Samaneh Azadi. Please find the paper at ArXiv or the International Journal of Computer Vision 2020.

Prerequisites:

  • Linux or macOS
  • Python 3.6
  • CPU or NVIDIA GPU + CUDA CuDNN

Preparation

Installation

  • Install PyTorch 1.0 and dependencies from http://pytorch.org
  • Install Torch vision from the source:
git clone https://github.com/pytorch/vision
cd vision
python setup.py install
  • Install visdom python library:
pip install visdom
  • Clone this repo:
git clone https://github.com/azadis/CompositionalGAN
cd CompositionalGAN

Datasets

  • For each pair of objects ${obj1_obj2} in {chair_table, basket_bottle, city_car, face_sunglasses}, download the dataset by:
bash scripts/${obj1_obj2}/download_data.sh

Individual chairs and tables are taken from Shapenet dataset, faces from CelebA dataset, and street scenes from Cityscapes.

Training

Viewpoint Transformation module:

If your model includes viewpoint transformation as in the chair_table experiment, train the Appearance Flow Network (AFN) by:

bash scripts/chair_table/train_AFN_Compose.sh

or download our trained AFN model:

bash scripts/chair_table/download_ckpt.sh

Paired Data

  • To train a compositional GAN model in order to compose each pair of objects ${obj1_obj2} given a paired training data, do:
bash scripts/${obj1_obj2}/train_objCompose_paired.sh
  • Before launching the above training script, set display_port to an arbitrary port number ${port} in the bash file and start the visdom server python -m visdom.server -p ${port}.

Unpaired Data

  • To train a model with unpaired training data, follow the same steps as above:
 scripts/${obj1_obj2}/train_objCompose_unpaired.sh

Testing

  • To download our trained models on each binary composition task of ${obj1_obj2}:
bash scripts/${obj1_obj2}/download_ckpt.sh
  • To test your trained model or the above downloaded checkpoints, run
bash scripts/${obj1_obj2}/test_objCompose_paired.sh

or

bash scripts/${obj1_obj2}/test_objCompose_unpaired.sh
  • Before launching the above scripts, set display_port to an arbitrary port number ${port} in the bash file and start the visdom server python -m visdom.server -p ${port}.

Visualization

  • To visualize the test results for the ${obj1_obj2} paired training after ${epoch} epochs:
  • Before ESMR
cd results/${obj1_obj2}_test_paired_compGAN/test_${epoch}/
python -m http.server 8884
  • After ESMR
cd results/finetune/${obj1_obj2}_test_paired_compGAN/test_${epoch}/
python -m http.server 8884
  • Replace paired with unpaired if you are training under the latter scenario.
  • Then in your local machine: ssh -N -f -L localhost:8884:localhost:8884 remote_user@remote_host

Citation

If you use this code or our compositional dataset, please cite our paper:

@article{azadi2018compositional,
  title={Compositional gan: Learning image-conditional binary composition},
  author={Azadi, Samaneh and Pathak, Deepak and Ebrahimi, Sayna and Darrell, Trevor},
  journal={arXiv preprint arXiv:1807.07560},
  year={2018}
}