CorresPondenceNet (CPNet)

This repository includes implementation and dataset for the ECCV '20 paper: "Human Correspondence Consensus for 3D Object Semantic Understanding". You can access the full paper from here or arXiv.

Dataset

CPNet dataset has a collection of 25 categories, 2,334 models based on ShapeNetCore, which includes 1,000+ correspondence sets with 104,861 points. The correspondence sets are shwon in figure below. Each correspondence contains points with same semantics on each object.

Dataset can be downloaded from Google Drive or OneDrive.

Data Layout

CPNet dataset contains the correspondence annotations on each object and geodesic distances between correspondences. The structure of dataset is as follow:

<data_path>/
    name2id.json
    <class_id>.h5
    corr_mean_dist_geo/
        <class_id>_mean_distances.h5

The details of each file is as follows:

• name2id.json: The dictionary between name and id of each class

• <class_id>.h5

  • "point_clouds": point coordinates of point cloud
  • "keypoints": point indexes of different correspondences
  • "mesh_names": names of meshes in ShapeNet

• <class_id>_mean_distances.h5

  • "mean_distance": distances between correspondences

How to Learn Dense Semantic Embeddings

Install

# Create a new conda environment
conda create -n cpnet python=3.7
conda activate cpnet

conda install pytorch==1.7.1 torchvision==0.8.2 cudatoolkit=10.1 -c pytorch
pip install -r requirements.txt

Data

Please add the absolute path of the CPNet dataset in config at first.

Train a Model

You can train on CPNet and get pointwise semantic embeddings with different backbones in models, for example:

python train.py network=pointnet2 class_name=airplane

Test with mGE

If you want to test the trained model for correspondence benchmark, please run

python test.py network=pointnet2 class_name=airplane

Cite CPNet

If you use CPNet in your research, please cite the paper:

@inproceedings{lou2020human,
    title={Human Correspondence Consensus for 3D Object Semantic Understanding},
    author={Lou, Yujing and You, Yang and Li, Chengkun and Cheng, Zhoujun and Li, Liangwei and Ma, Lizhuang and Wang, Weiming and Lu, Cewu},
    booktitle={European Conference on Computer Vision},
    pages={496--512},
    year={2020},
    organization={Springer}
}

References

The implementations of models are referenced by:

• yanx27/Pointnet_Pointnet2_pytorch
• WangYueFt/dgcnn
• DylanWusee/pointconv_pytorch
• sausagecy/RSCNN_Pytorch1.0
• qianguih/RSNet
• xyf513/SpiderCNN