Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COLMAP feature_extractor crashing on ns-process-data video call #3539

Open
chambbj opened this issue Dec 3, 2024 · 1 comment
Open

COLMAP feature_extractor crashing on ns-process-data video call #3539

chambbj opened this issue Dec 3, 2024 · 1 comment

Comments

@chambbj
Copy link

chambbj commented Dec 3, 2024

Describe the bug
The COLMAP feature_extractor is reporting an error when running ns-process-data video on the v1.1.5 release tag (and main branch) with a pixi install of nerfstudio running on a LambdaLabs A10 instance.

To Reproduce
Steps to reproduce the behavior:

  1. Start gpu_1x_a10 instance on cloud.lambdalabs.com. Launch cloud IDE and terminal window.
  2. Install pixi
curl -fsSL https://pixi.sh/install.sh | bash
  1. Install nerfstudio from v1.1.5 tag (note that this also happened on the main branch)
git clone https://github.com/nerfstudio-project/nerfstudio.git
cd nerfstudio
git checkout tags/v1.1.5
pixi run post-install
pixi shell

yields

$ pixi run post-install
 WARN The package `requests==2.31.0` does not have an extra named `socks`
✨ Pixi task (make-third_party-dir): ls third_party || mkdir third_party
Hierarchical-Localization

✨ Pixi task (clone-hloc): ls Hierarchical-Localization || git clone --recursive https://github.com/cvg/Hierarchical-Localization/
Dockerfile  README.md  demo.ipynb  hloc           pairs                  pipeline_InLoc.ipynb  requirements.txt  third_party
LICENSE     datasets   doc         hloc.egg-info  pipeline_Aachen.ipynb  pipeline_SfM.ipynb    setup.py

✨ Pixi task (hloc-install): python -m pip install -e .
Task 'hloc-install' can be skipped (cache hit) 🚀

✨ Pixi task (tcnn-install): python -c 'import tinycudann as tcnn' || python -m pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

✨ Pixi task (post-install): pwd
/home/ubuntu/gaussian-splatting/nerfstudio-repo
$ pixi shell
 WARN The package `requests==2.31.0` does not have an extra named `socks`
  1. Process video
(nerfstudio) $ ns-process-data video --data video.mp4 --output-dir processed-video-300
Number of frames in video: 5915
Extracting 312 frames in evenly spaced intervals
[16:44:54] 🎉 Done converting video to images.                                                 process_data_utils.py:227+

leads to

──────────────────────────────────────────────  💀 💀 💀 ERROR 💀 💀 💀  ───────────────────────────────────────────────
Error running command: colmap feature_extractor --database_path processed-video-300/colmap/database.db --image_path 
processed-video-300/images --ImageReader.single_camera 1 --ImageReader.camera_model OPENCV --SiftExtraction.use_gpu 1
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
QObject::moveToThread: Current thread (0x59b2cd740780) is not the object's thread (0x59b2cd74f190).
Cannot move to target thread (0x59b2cd740780)

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in 
"/home/ubuntu/gaussian-splatting/nerfstudio-repo/.pixi/envs/default/lib/python3.10/site-packages/cv2/qt/plugins" even 
though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may 
fix this problem.

Available platform plugins are: xcb, eglfs, minimal, minimalegl, offscreen, vnc, webgl.

*** Aborted at 1733244295 (unix time) try "date -d @1733244295" if you are using GNU date ***
PC: @     0x7884232969fc pthread_kill
*** SIGABRT (@0x3e800003288) received by PID 12936 (TID 0x78841f6d70c0) from PID 12936; stack trace: ***
    @     0x788423299ee8 (unknown)
    @     0x78842567b917 google::(anonymous namespace)::FailureSignalHandler(int, siginfo*, void*)
    @     0x788423242520 (unknown)
    @     0x7884232969fc pthread_kill
    @     0x788423242476 raise
    @     0x7884232287f3 abort
    @     0x788423699ad1 QMessageLogger::fatal(char const*, ...) const
    @     0x788423f2c4d3 QGuiApplicationPrivate::createPlatformIntegration()
    @     0x788423f2c9b0 QGuiApplicationPrivate::createEventDispatcher()
    @     0x7884238ba2f5 QCoreApplicationPrivate::init()
    @     0x788423f2f359 QGuiApplicationPrivate::init()
    @     0x788424775309 QApplicationPrivate::init()
    @     0x59b2cb59d166 colmap::RunFeatureExtractor(int, char**)
    @     0x59b2cb590fa9 main
    @     0x788423229d90 (unknown)
    @     0x788423229e40 __libc_start_main
    @     0x59b2cb5940a4 (unknown)
Aborted

Expected behavior
I would expect the COLMAP feature_extractor to run.

Screenshots
N/A

Additional context
If I disable the GPU, I am fine, as demonstrated below.

(nerfstudio) $ ns-process-data video --data video.mp4 --output-dir processed-video-300 --no-gpu
Number of frames in video: 5915
Extracting 312 frames in evenly spaced intervals
[16:45:33] 🎉 Done converting video to images.                                                 process_data_utils.py:227
[16:45:39] 🎉 Done extracting COLMAP features.                                                       colmap_utils.py:137
[16:46:49] 🎉 Done matching COLMAP features.                                                         colmap_utils.py:151
[16:50:06] 🎉 Done COLMAP bundle adjustment.                                                         colmap_utils.py:173
[16:51:44] 🎉 Done refining intrinsics.                                                              colmap_utils.py:184
[16:51:45] 🎉 🎉 🎉 All DONE 🎉 🎉 🎉                                                 video_to_nerfstudio_dataset.py:149
           Starting with 5915 video frames                                            video_to_nerfstudio_dataset.py:152
           We extracted 312 images with prefix 'frame_'                               video_to_nerfstudio_dataset.py:152
           Colmap matched 295 images                                                  video_to_nerfstudio_dataset.py:152
           COLMAP found poses for 94.55% of the images.                               video_to_nerfstudio_dataset.py:152
@yuzewang1998
Copy link

same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants