Neural Mesh-Based Graphics

Shubhendu Jena, Franck Multon, Adnane Boukhayma.
ECCV Workshop 2022 CV4Metaverse

Install

Run this command to install python environment using conda:

source scripts/install_deps.sh

Please modify the cudatoolkit version in the bash script above according to your system compatibilities.

Data Preparation

Download the Tanks and Temples data as in FreeViewSynthesis.
The data directory structure should follow the below hierarchy.

Neural-Mesh-Based-Graphics  
|-- data  
|   |-- ibr3d_tat
|   |-- split_meshes
|   |-- Checkpoints_Truck
|   |-- Checkpoints_Train
|   |-- Checkpoints_Playground
|   |-- Checkpoints_M60

To download split meshes for all scenes, see Split Meshes.
To copy the split and processed meshes, run

python scripts/copy_t_and_t.py

Pretrained Models/Checkpoints

Pretrained models/Checkpoints can be found here.

Evaluation

To start the evaluation please run the following command:

python train.py --config configs/test_example.yaml --pipeline nmbg.pipelines.p3d.TexturePipeline --eval

Training

To start a single scene or full training please run the following command:

python train.py --config configs/train_example.yaml --pipeline nmbg.pipelines.p3d.TexturePipeline --train

with the appropriate paths_file (configs/paths_example_small_train.yaml for single scene and configs/paths_example_full_train.yaml for full training) in train_example.yaml.

Acknowledgements

Parts of the code were based on the original Neural Point-Based Graphics implementation, Stable View Synthesis implementation and NeRF-SH implementation.