Skip to content

Latest commit

 

History

History
80 lines (72 loc) · 3.23 KB

README.md

File metadata and controls

80 lines (72 loc) · 3.23 KB
Raw

TULIP: Transformer for Upsampling of LiDAR Point Clouds

This is an official implementation of the paper TULIP: Transformer for Upsampling of LiDAR Point Clouds: A framework for LiDAR upsampling using Swin Transformer (CVPR2024)

Demo

The visualization is done by sampling a time-series subset from the test split

KITTI DurLAR CARLA
KITTI DurLAR CARLA

Installation

Our work is implemented with the following environmental setups:

  • Python == 3.8
  • PyTorch == 1.12.0
  • CUDA == 11.3

You can use conda to create the correct environment:

conda create -n myenv python=3.8
conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=11.3 -c pytorch

Then, install the dependencies in the environment:

pip install -r requirements.txt
pip install git+'https://github.com/otaheri/chamfer_distance'  # need access to gpu for compilation

You can refer to more details about chamfer distance package from https://github.com/otaheri/chamfer_distance

Data Preparation

We have evaluated our method on three different datasets and they are all open source datasets:

After downloading the raw dataset, create train and test split for LiDAR upsampling:

bash bash_scripts/create_durlar_dataset.sh
bash bash_scripts/create_kitti_dataset.sh

The new dataset should be structured in this way:

dataset
│
└───KITTI / DurLAR
   │
   └───train
   │   │   00000001.npy
   │   │   00000002.npy
   │   │   ...
   └───val
       │   00000001.npy
       │   00000002.npy
       │   ...

Training

We provide some bash files for running the experiment quickly with default settings.

bash bash_scripts/tulip_upsampling_kitti.sh (KITTI)
bash bash_scripts/tulip_upsampling_carla.sh (CARLA)
bash bash_scripts/tulip_upsampling_durlar.sh (DurLAR)

Evaluation

You can download the pretrained models from the link and use them for evaluation.

bash bash_scripts/tulip_evaluation_kitti.sh (KITTI)
bash bash_scripts/tulip_evaluation_carla.sh (CARLA)
bash bash_scripts/tulip_evaluation_durlar.sh (DurLAR)

Citation

@inproceedings{yang2024tulip,
  title={TULIP: Transformer for Upsampling of LiDAR Point Clouds},
  author={Yang, Bin and Pfreundschuh, Patrick and Siegwart, Roland and Hutter, Marco and Moghadam, Peyman and Patil, Vaishakh},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={15354--15364},
  year={2024}
}