Submitted to IEEE International Conference on Robotics & Automation (ICRA) 2025
This repo houses the core code for Next Best Sense. We exclusively use Docker for this work, which allows for self-contained code that won't mess up the host OS.
The pipeline has been tested on Ubuntu 22.04. To avoid installation pains and dependency conflicts, we have a publicly available Dockerfile that includes everything here.
To pull from Docker, run
docker pull peasant98/active-touch-gs:latest
- CUDA 11+ and a GPU with at least 16GB VRAM
- Python 3.8+
- ROS1 Noetic
- Conda or Mamba (optional)
- Kinova Gen3 robot (7 DoF)
Install PyTorch with CUDA (this repo has been tested with CUDA 11.8 and CUDA 12.1).
For CUDA 11.8:
conda create --name touch-gs python=3.8
conda activate touch-gs
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit
See Dependencies in the Installation documentation for more.
Repo Cloning This repository should be used as a group of packages for a ROS1 workspace for controlling a Kinova arm.
We note that we have our own version of Nerfstudio, which supports active learning, which can be found here.
To install, the steps are:
git clone https://github.com/JiangWenPL/FisherRF-ns
cd FisherRF-ns
# install the package in editable mode for easy
python3 -m pip install -e . -v
First, build the workspace:
# be outside the NextBestSense dir (ROS workspace)
catkin build
source install/setup.bash
Then, run the launch file, which will open the controller and vision node. Assuming you have our version of Nerfstudio installed, this will work as follows:
- Run Kinova pipeline.
roslaunch kinova_control moveit_controller.launch
We have made an end-to-end pipeline that will take care of setting up the data, training, and evaluating our method. Note that we will release the code for running the ablations (which includes the baselines) soon!
You can get rendered videos with a custom camera path detailed here. This is how we were able to get our videos on our website.