diff --git a/README.md b/README.md index 0bade9b9d..81340e5a2 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,7 @@ |:------------:| :-------------:| | Ubuntu 20.04.3 | ![Linux CI](https://github.com/borglab/gtsfm/actions/workflows/test-python.yml/badge.svg?branch=master) | -Georgia Tech Structure-from-Motion (GTSfM) is an end-to-end SfM pipeline based on [GTSAM](https://github.com/borglab/gtsam). GTSfM was designed from the ground-up to natively support parallel computation using [Dask](https://dask.org/). - -For more details, please refer to our [arXiv preprint](https://arxiv.org/abs/2311.18801). +### Georgia Tech Structure-from-Motion (GTSfM) is an end-to-end SfM pipeline based on [GTSAM](https://github.com/borglab/gtsam). GTSfM was designed from the ground-up to natively support parallel computation using [Dask](https://dask.org/).

@@ -24,6 +22,8 @@ The majority of our code is governed by an MIT license and is suitable for comme ## Installation +

Click to expand + GTSfM requires no compilation, as Python wheels are provided for GTSAM. This repository includes external repositories as Git submodules –- don't forget to pull submodules with `git submodule update --init --recursive` or clone with `git clone --recursive https://github.com/borglab/gtsfm.git`. To run GTSfM, first, we need to create a conda environment with the required dependencies. @@ -50,7 +50,11 @@ pip install -e . Make sure that you can run `python -c "import gtsfm; import gtsam; print('hello world')"` in python, and you are good to go! -## Usage Guide (Running 3D Reconstruction) +
+ +## Usage Guide + +
Click to expand Before running reconstruction, if you intend to use modules with pre-trained weights, such as SuperPoint, SuperGlue, or PatchmatchNet, please first run: @@ -107,19 +111,19 @@ For users that want to run GTSfM on a cluster of multiple machines, we provide s The results will be stored at `--output_root`, which is the `results` folder in the repo root by default. The poses and 3D tracks are stored in COLMAP format inside the `ba_output` subdirectory of `--output_root`. These can be visualized using the COLMAP GUI as well. -### Nerfstudio +
-We provide a preprocessing script to convert the camera poses estimated by GTSfM to [nerfstudio](https://docs.nerf.studio/en/latest/) format: +## Pipeline Overview -```bash -python scripts/prepare_nerfstudio.py --results_path {RESULTS_DIR} --images_dir {IMAGES_DIR} -``` +![Alt text](assets/gtsfm-overview.svg?raw=true) -The results are stored in the nerfstudio_input subdirectory inside `{RESULTS_DIR}`, which can be used directly with nerfstudio if installed: +GTSfM attempts to make the global Structure-from-Motion process as modular as possible to allow for streamlined integration of new state-of-the-art tools. We provide details for each module of the GTSfM pipeline below. -```bash -ns-train nerfacto --data {RESULTS_DIR}/nerfstudio_input -``` +- [Loader](assets/LOADER.md) +- [Image Pairs Generator](assets/IMAGE_PAIRS_GENERATOR.md) +- [Correspondence Generator](assets/CORRESPONDENCE_GENERATOR.md) +- [Two View Estimator](assets/TWO_VIEW_ESTIMATOR.md) +- [Multiview Optimizer](assets/MULTIVIEW_OPTIMIZER.md) ## Repository Structure @@ -143,6 +147,20 @@ GTSfM is designed in an extremely modular way. Each module can be swapped out wi - `utils`: utility functions such as serialization routines and pose comparisons, etc - `tests`: unit tests on every function and module +## Nerfstudio + +We provide a preprocessing script to convert the camera poses estimated by GTSfM to [nerfstudio](https://docs.nerf.studio/en/latest/) format: + +```bash +python scripts/prepare_nerfstudio.py --results_path {RESULTS_DIR} --images_dir {IMAGES_DIR} +``` + +The results are stored in the nerfstudio_input subdirectory inside `{RESULTS_DIR}`, which can be used directly with nerfstudio if installed: + +```bash +ns-train nerfacto --data {RESULTS_DIR}/nerfstudio_input +``` + ## Contributing Contributions are always welcome! Please be aware of our [contribution guidelines for this project](CONTRIBUTING.md). diff --git a/assets/CORRESPONDENCE_GENERATOR.md b/assets/CORRESPONDENCE_GENERATOR.md new file mode 100644 index 000000000..a1ac8d7d6 --- /dev/null +++ b/assets/CORRESPONDENCE_GENERATOR.md @@ -0,0 +1,177 @@ +# Correspondence Generator + +![Alt text](gtsfm-overview-correspondence-generator.svg?raw=true) + +- [Loader](assets/LOADER.md) +- [Image Pairs Generator](assets/IMAGE_PAIRS_GENERATOR.md) +- **Correspondence Generator** +- [Two View Estimator](assets/TWO_VIEW_ESTIMATOR.md) +- [Multiview Optimizer](assets/MULTIVIEW_OPTIMIZER.md) + +## What is a Correspondence Generator? + +The Correspondence Generator is responsible for taking in putative image pairs from the [`ImagePairsGenerator`](https://github.com/borglab/gtsfm/blob/master/gtsfm/retriever/image_pairs_generator.py) and returning keypoints for each image and correspondences between each specified image pair. Correspondence generation is implemented by the [`CorrespondenceGeneratorBase`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/correspondence_generator/correspondence_generator_base.py) class defined below. + +```python +class CorrespondenceGeneratorBase: + """Base class for correspondence generators.""" + + @abstractmethod + def generate_correspondences( + self, + client: Client, + images: List[Future], + image_pairs: List[Tuple[int, int]], + ) -> Tuple[List[Keypoints], Dict[Tuple[int, int], np.ndarray]]: + """Apply the correspondence generator to generate putative correspondences. + + Args: + client: Dask client, used to execute the front-end as futures. + images: List of all images, as futures. + image_pairs: Indices of the pairs of images to estimate two-view pose and correspondences. + + Returns: + List of keypoints, one entry for each input images. + Putative correspondence as indices of keypoints, for pairs of images. + """ +``` + +## Types of Correspondence Generators + +We provide support for two correspondence generation paradigms: [feature extraction _then_ matching](#feature-detection-and-description-then-matching) and [_detector-free_ matching](#detector-free-matching). The supported Correspondence Generator paradigms are detailed below. + + +### Feature Detection and Description _then_ Matching + +This paradigm jointly computes feature detections and descriptors, typically using shared weights in a deep convolutional neural network, followed by feature matching. This is implemented by the [`DetDescCorrespondenceGenerator`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/correspondence_generator/det_desc_correspondence_generator.py) class, which wraps a feature detector and descriptor ([`DetectorDescriptorBase`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/detector_descriptor/detector_descriptor_base.py)) and a feature matcher ([`MatcherBase`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/matcher/matcher_base.py)). + +The feature detector and descriptor module takes in a single image and outputs keypoints and feature descriptors. Joint detection and description is implemented by the [`DetectorDescriptorBase`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/detector_descriptor/detector_descriptor_base.py) class defined below. We also provide functionality for combining different keypoint detection and feature description modules to form a joint detector-descriptor module (see [`CombinationDetectorDescriptor`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/detector_descriptor/combination_detector_descriptor.py)). To create your own feature extractor, simply copy the contents of `detector_descriptor_base.py` to a new file corresponding to the new extractor's class name and implement the `detect_and_describe` method. + +The feature matcher takes in the keypoints and descriptors for image images and outputs indices for matching keypoints. Feature matching is implemented by the [`MatcherBase`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/matcher/matcher_base.py) class defined below. To create your own feature matcher, simply copy the contents of `matcher_base.py` to a new file corresponding to the new matcher's class name and implement the `match` method. + +```python +class DetectorDescriptorBase(GTSFMProcess): + """Base class for all methods which provide a joint detector-descriptor to work on a single image.""" + + def __init__(self, max_keypoints: int = 5000): + """Initialize the detector-descriptor. + + Args: + max_keypoints: Maximum number of keypoints to detect. Defaults to 5000. + """ + self.max_keypoints = max_keypoints + + @abc.abstractmethod + def detect_and_describe(self, image: Image) -> Tuple[Keypoints, np.ndarray]: + """Perform feature detection as well as their description. + + Refer to detect() in DetectorBase and describe() in DescriptorBase for + details about the output format. + + Args: + image: the input image. + + Returns: + Detected keypoints, with length N <= max_keypoints. + Corr. descriptors, of shape (N, D) where D is the dimension of each descriptor. + """ +``` + +```python +class MatcherBase(GTSFMProcess): + """Base class for all matchers.""" + + @abc.abstractmethod + def match( + self, + keypoints_i1: Keypoints, + keypoints_i2: Keypoints, + descriptors_i1: np.ndarray, + descriptors_i2: np.ndarray, + im_shape_i1: Tuple[int, int, int], + im_shape_i2: Tuple[int, int, int], + ) -> np.ndarray: + """Match descriptor vectors. + + # Some matcher implementations (such as SuperGlue) utilize keypoint coordinates as + # positional encoding, so our matcher API provides them for optional use. + + Output format: + 1. Each row represents a match. + 2. First column represents keypoint index from image #i1. + 3. Second column represents keypoint index from image #i2. + 4. Matches are sorted in descending order of the confidence (score), if possible. + + Args: + keypoints_i1: keypoints for image #i1, of length N1. + keypoints_i2: keypoints for image #i2, of length N2. + descriptors_i1: descriptors corr. to keypoints_i1. + descriptors_i2: descriptors corr. to keypoints_i2. + im_shape_i1: shape of image #i1, as (height,width,channel). + im_shape_i2: shape of image #i2, as (height,width,channel). + + + Returns: + Match indices (sorted by confidence), as matrix of shape (N, 2), where N < min(N1, N2). + """ +``` + +
Supported Feature Detectors & Descriptors + +
+ +
Supported Feature Matchers + +
+ + +### Detector-Free Matching + +This paradigm directly regresses _per-pixel_ matches between two input images as opposed to generating detections for each image followed by matching. This is implemented by the [`ImageCorrespondenceGenerator`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/correspondence_generator/image_correspondence_generator.py) class, which siply wraps an detector-free matcher ([`ImageMatcherBase`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/matcher/image_matcher_base.py)). + +Detector-free matching is implemented by the `ImageMatcherBase` class defined below. To create your own detector-free matcher, simply copy the contents of `image_matcher_base.py` to a new file corresponding to the new matcher's class name and implement the `match` method. + +```python +class ImageMatcherBase(GTSFMProcess): + """Base class for matchers that accept an image pair, and directly generate keypoint matches. + + Note: these matchers do NOT use descriptors as input. + """ + + @abc.abstractmethod + def match( + self, + image_i1: Image, + image_i2: Image, + ) -> Tuple[Keypoints, Keypoints]: + """Identify feature matches across two images. + + Args: + image_i1: first input image of pair. + image_i2: second input image of pair. + + Returns: + Keypoints from image 1 (N keypoints will exist). + Corresponding keypoints from image 2 (there will also be N keypoints). These represent feature matches. + """ +``` + +
Supported Detector-Free Matchers + +
diff --git a/assets/IMAGE_PAIRS_GENERATOR.md b/assets/IMAGE_PAIRS_GENERATOR.md new file mode 100644 index 000000000..1f215c67e --- /dev/null +++ b/assets/IMAGE_PAIRS_GENERATOR.md @@ -0,0 +1,115 @@ +# Image Pairs Generator + +![Alt text](gtsfm-overview-image-pairs-generator.svg?raw=true) + +- [Loader](assets/LOADER.md) +- **Image Pairs Generator** +- [Correspondence Generator](assets/CORRESPONDENCE_GENERATOR.md) +- [Two View Estimator](assets/TWO_VIEW_ESTIMATOR.md) +- [Multiview Optimizer](assets/MULTIVIEW_OPTIMIZER.md) + +## What is an Image Pair Generator? + +The Image Pair Generator takes in images from the Loader and outputs putative image pairs for correspondence generation. Image pair generation is implemented by the [`ImagePairsGenerator`](https://github.com/borglab/gtsfm/blob/master/gtsfm/retriever/image_pairs_generator.py) class defined below, which wraps a specific [`Retriever`](https://github.com/borglab/gtsfm/blob/master/gtsfm/retriever/retriever_base.py), and, optionally, a [`GlobalDescriptor`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/global_descriptor/global_descriptor_base.py). + +We support exhaustive, sequential, and descriptor-based image pair generation: +- **Exhaustive** matching compares every image pair, ensuring all possible matches are found but at a high computational cost. +- **Sequential** matching only compares neighboring images in a sequence (according to their image ID), making it faster but missing potentially valuable non-adjacent image pairs. +- **Descriptor-based** matching leverages global images descriptors to pre-filter and identify images pairs that have a higher probability of overlap, balancing efficiency with accuracy. This is discussed further in the [following section](#global-image-descriptors). + + +```python +class ImagePairsGenerator: + def __init__(self, retriever: RetrieverBase, global_descriptor: Optional[GlobalDescriptorBase] = None): + self._global_descriptor: Optional[GlobalDescriptorBase] = global_descriptor + self._retriever: RetrieverBase = retriever + + def __repr__(self) -> str: + return f""" + ImagePairGenerator: + {self._global_descriptor} + {self._retriever} + """ + + def generate_image_pairs( + self, client: Client, images: List[Future], image_fnames: List[str], plots_output_dir: Optional[Path] = None + ) -> List[Tuple[int, int]]: + def apply_global_descriptor(global_descriptor: GlobalDescriptorBase, image: Image) -> np.ndarray: + return global_descriptor.describe(image=image) + + descriptors: Optional[List[np.ndarray]] = None + if self._global_descriptor is not None: + global_descriptor_future = client.scatter(self._global_descriptor, broadcast=False) + + descriptor_futures = [ + client.submit(apply_global_descriptor, global_descriptor_future, image) for image in images + ] + + descriptors = client.gather(descriptor_futures) + + return self._retriever.get_image_pairs( + global_descriptors=descriptors, image_fnames=image_fnames, plots_output_dir=plots_output_dir + ) +``` + +## Global Image Descriptors + +Global desriptors work similar to local feature desriptors except that these methods generate a single descriptor for each image. Distances between these global image descriptors can then be used as a metric for the expected "matchability" of the image pairs during the correspondence generation phase, where a threshold can be used to reject potentially dissimilar image pairs before conducting correspondence generation. This reduces the likelihood of matching image pairs with little to no overlap that could cause erroneous correspondences to be inserted into the optimization process in the back-end while also significantly reducing the runtimes as compared to exhaustive matching. + +Global descriptor modules are implemented following the [`GlobalDescriptorBase`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/global_descriptor/global_descriptor_base.py) class and must be wrapped using a corresponding [`RetrieverBase`](https://github.com/borglab/gtsfm/blob/master/gtsfm/retriever/retriever_base.py) implementation, where the global descriptor module takes in individual images and outputs their corresponding descriptor and the retriever module takes these descriptors descriptors and computes the image pair similarity scores and outputs the putative image pairs based on a specified threshold (see [`NetVLADGlobalDescriptor`](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/global_descriptor/netvlad_global_descriptor.py) and [`NetVLADRetriever`](https://github.com/borglab/gtsfm/blob/master/gtsfm/retriever/netvlad_retriever.py)). + +```python +class RetrieverBase(GTSFMProcess): + """Base class for image retriever implementations.""" + + def __init__(self, matching_regime: ImageMatchingRegime) -> None: + """ + Args: + matching_regime: identifies type of matching used for image retrieval, e.g., exhaustive, descriptor-based. + """ + self._matching_regime = matching_regime + + @abc.abstractmethod + def get_image_pairs( + self, + global_descriptors: Optional[List[np.ndarray]], + image_fnames: List[str], + plots_output_dir: Optional[Path] = None, + ) -> List[Tuple[int, int]]: + """Compute potential image pairs. + + Args: + global_descriptors: the global descriptors for the retriever, if needed. + image_fnames: file names of the images + plots_output_dir: Directory to save plots to. If None, plots are not saved. + + Returns: + List of (i1,i2) image pairs. + """ +``` + +```python +class GlobalDescriptorBase: + """Base class for all the global image descriptors. + + Global image descriptors assign a vector for each input image. + """ + + @abc.abstractmethod + def describe(self, image: Image) -> np.ndarray: + """Compute the global descriptor for a single image query. + + Args: + image: input image. + + Returns: + img_desc: array of shape (D,) representing global image descriptor. + """ + +``` + +
Supported Global Descriptors + +
diff --git a/assets/LOADER.md b/assets/LOADER.md new file mode 100644 index 000000000..e73721255 --- /dev/null +++ b/assets/LOADER.md @@ -0,0 +1,11 @@ +# Loader + +![Alt text](gtsfm-overview-loader.svg?raw=true) + +- **Loader** +- [Image Pairs Generator](assets/IMAGE_PAIRS_GENERATOR.md) +- [Correspondence Generator](assets/CORRESPONDENCE_GENERATOR.md) +- [Two View Estimator](assets/TWO_VIEW_ESTIMATOR.md) +- [Multiview Optimizer](assets/MULTIVIEW_OPTIMIZER.md) + +## What is a Loader? \ No newline at end of file diff --git a/assets/MULTIVIEW_OPTIMIZER.md b/assets/MULTIVIEW_OPTIMIZER.md new file mode 100644 index 000000000..32534c528 --- /dev/null +++ b/assets/MULTIVIEW_OPTIMIZER.md @@ -0,0 +1,12 @@ +# Multiview Optimizer + +![Alt text](gtsfm-overview-two-view-estimator.svg?raw=true) + +- [Loader](assets/LOADER.md) +- [Image Pairs Generator](assets/IMAGE_PAIRS_GENERATOR.md) +- [Correspondence Generator](assets/CORRESPONDENCE_GENERATOR.md) +- [Two View Estimator](assets/TWO_VIEW_ESTIMATOR.md) +- **Multiview Optimizer** + +## What is a Multiview Optimizer? + diff --git a/assets/TWO_VIEW_ESTIMATOR.md b/assets/TWO_VIEW_ESTIMATOR.md new file mode 100644 index 000000000..777da208a --- /dev/null +++ b/assets/TWO_VIEW_ESTIMATOR.md @@ -0,0 +1,11 @@ +# Two View Estimator + +![Alt text](gtsfm-overview-two-view-estimator.svg?raw=true) + +- [Loader](assets/LOADER.md) +- [Image Pairs Generator](assets/IMAGE_PAIRS_GENERATOR.md) +- [Correspondence Generator](assets/CORRESPONDENCE_GENERATOR.md) +- **Two View Estimator** +- [Multiview Optimizer](assets/MULTIVIEW_OPTIMIZER.md) + +## What is a Two-View Estimator? \ No newline at end of file diff --git a/assets/gtsfm-overview-correspondence-generator.svg b/assets/gtsfm-overview-correspondence-generator.svg new file mode 100644 index 000000000..a0734d457 --- /dev/null +++ b/assets/gtsfm-overview-correspondence-generator.svg @@ -0,0 +1,4 @@ + + + +
Correspondence Generator
Detector
Descriptor
Matcher
"Detector-Free" Matcher
Detector + Descriptor
Matcher
Image Pairs Generator
Global Descriptor
Retriever
Two-View Estimator
Verifier
Inlier Support Processor
Image pairs
Keypoints
Relative poses
Multiview Optimizer
View Graph Estimation
Translation Averaging
Rotation Averaging
Triangulation
Bundle Adjustment
Putative correspondences
Keypoints
Verified correspondences
Initial global poses
Keypoints
Verified correspondences
View graph
Landmark
Loader
Images
Camera intrinsics (optional)
Camera extrinsics (optional)
 \ No newline at end of file diff --git a/assets/gtsfm-overview-image-pairs-generator.svg b/assets/gtsfm-overview-image-pairs-generator.svg new file mode 100644 index 000000000..8e763a3fc --- /dev/null +++ b/assets/gtsfm-overview-image-pairs-generator.svg @@ -0,0 +1,4 @@ + + + +
Correspondence Generator
Detector
Descriptor
Matcher
"Detector-Free" Matcher
Detector + Descriptor
Matcher
Image Pairs Generator
Global Descriptor
Retriever
Two-View Estimator
Verifier
Inlier Support Processor
Image pairs
Keypoints
Relative poses
Multiview Optimizer
View Graph Estimation
Translation Averaging
Rotation Averaging
Triangulation
Bundle Adjustment
Putative correspondences
Keypoints
Verified correspondences
Initial global poses
Keypoints
Verified correspondences
View graph
Landmark
Loader
Images
Camera intrinsics (optional)
Camera extrinsics (optional)
 \ No newline at end of file diff --git a/assets/gtsfm-overview-loader.svg b/assets/gtsfm-overview-loader.svg new file mode 100644 index 000000000..acc16410b --- /dev/null +++ b/assets/gtsfm-overview-loader.svg @@ -0,0 +1,4 @@ + + + +
Correspondence Generator
Detector
Descriptor
Matcher
"Detector-Free" Matcher
Detector + Descriptor
Matcher
Image Pairs Generator
Global Descriptor
Retriever
Two-View Estimator
Verifier
Inlier Support Processor
Image pairs
Keypoints
Relative poses
Multiview Optimizer
View Graph Estimation
Translation Averaging
Rotation Averaging
Triangulation
Bundle Adjustment
Putative correspondences
Keypoints
Verified correspondences
Initial global poses
Keypoints
Verified correspondences
View graph
Landmark
Loader
Images
Camera intrinsics (optional)
Camera extrinsics (optional)
 \ No newline at end of file diff --git a/assets/gtsfm-overview-multiview-optimizer.svg b/assets/gtsfm-overview-multiview-optimizer.svg new file mode 100644 index 000000000..113e8d6a1 --- /dev/null +++ b/assets/gtsfm-overview-multiview-optimizer.svg @@ -0,0 +1,4 @@ + + + +
Correspondence Generator
Detector
Descriptor
Matcher
"Detector-Free" Matcher
Detector + Descriptor
Matcher
Image Pairs Generator
Global Descriptor
Retriever
Two-View Estimator
Verifier
Inlier Support Processor
Image pairs
Keypoints
Relative poses
Multiview Optimizer
View Graph Estimation
Translation Averaging
Rotation Averaging
Triangulation
Bundle Adjustment
Putative correspondences
Keypoints
Verified correspondences
Initial global poses
Keypoints
Verified correspondences
View graph
Landmark
Loader
Images
Camera intrinsics (optional)
Camera extrinsics (optional)
 \ No newline at end of file diff --git a/assets/gtsfm-overview-two-view-estimator.svg b/assets/gtsfm-overview-two-view-estimator.svg new file mode 100644 index 000000000..0f3ab3bec --- /dev/null +++ b/assets/gtsfm-overview-two-view-estimator.svg @@ -0,0 +1,4 @@ + + + +
Correspondence Generator
Detector
Descriptor
Matcher
"Detector-Free" Matcher
Detector + Descriptor
Matcher
Image Pairs Generator
Global Descriptor
Retriever
Two-View Estimator
Verifier
Inlier Support Processor
Image pairs
Keypoints
Relative poses
Multiview Optimizer
View Graph Estimation
Translation Averaging
Rotation Averaging
Triangulation
Bundle Adjustment
Putative correspondences
Keypoints
Verified correspondences
Initial global poses
Keypoints
Verified correspondences
View graph
Landmark
Loader
Images
Camera intrinsics (optional)
Camera extrinsics (optional)
 \ No newline at end of file diff --git a/assets/gtsfm-overview.svg b/assets/gtsfm-overview.svg new file mode 100644 index 000000000..dd9022f43 --- /dev/null +++ b/assets/gtsfm-overview.svg @@ -0,0 +1,4 @@ + + + +
Correspondence Generator
Detector
Descriptor
Matcher
"Detector-Free" Matcher
Detector + Descriptor
Matcher
Image Pairs Generator
Global Descriptor
Retriever
Two-View Estimator
Verifier
Inlier Support Processor
Image pairs
Keypoints
Relative poses
Multiview Optimizer
View Graph Estimation
Translation Averaging
Rotation Averaging
Triangulation
Bundle Adjustment
Putative correspondences
Keypoints
Verified correspondences
Initial global poses
Keypoints
Verified correspondences
View graph
Landmark
Loader
Images
Camera intrinsics (optional)
Camera extrinsics (optional)
 \ No newline at end of file diff --git a/gtsfm/frontend/detector_descriptor/detector_descriptor_base.py b/gtsfm/frontend/detector_descriptor/detector_descriptor_base.py index 00d13e9ad..4c890891d 100644 --- a/gtsfm/frontend/detector_descriptor/detector_descriptor_base.py +++ b/gtsfm/frontend/detector_descriptor/detector_descriptor_base.py @@ -17,10 +17,7 @@ class DetectorDescriptorBase(GTSFMProcess): - """Base class for all methods which provide a joint detector-descriptor to work on an image. - - This class serves as a combination of individual detector and descriptor. - """ + """Base class for all methods which provide a joint detector-descriptor to work on a single image.""" def get_ui_metadata() -> UiMetadata: """Returns data needed to display node and edge info for this process in the process graph.""" diff --git a/gtsfm/frontend/detector_descriptor/superpoint.py b/gtsfm/frontend/detector_descriptor/superpoint.py index 8467472cc..e3fb4522e 100644 --- a/gtsfm/frontend/detector_descriptor/superpoint.py +++ b/gtsfm/frontend/detector_descriptor/superpoint.py @@ -4,7 +4,7 @@ by wrapping over the authors' implementation. References: -- https://arxiv.org/abs/1712.07629 +- https://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w9/DeTone_SuperPoint_Self-Supervised_Interest_CVPR_2018_paper.pdf - https://github.com/magicleap/SuperGluePretrainedNetwork Authors: Ayush Baid diff --git a/gtsfm/frontend/matcher/matcher_base.py b/gtsfm/frontend/matcher/matcher_base.py index 43d1a8be3..da1f1dc63 100644 --- a/gtsfm/frontend/matcher/matcher_base.py +++ b/gtsfm/frontend/matcher/matcher_base.py @@ -12,10 +12,7 @@ class MatcherBase(GTSFMProcess): - """Base class for all matchers. - - Matchers work on a pair of descriptors and match them by their distance. - """ + """Base class for all matchers.""" def get_ui_metadata() -> UiMetadata: """Returns data needed to display node and edge info for this process in the process graph.""" diff --git a/gtsfm/retriever/retriever_base.py b/gtsfm/retriever/retriever_base.py index ef3839517..5b6c481fe 100644 --- a/gtsfm/retriever/retriever_base.py +++ b/gtsfm/retriever/retriever_base.py @@ -16,7 +16,7 @@ class ImageMatchingRegime(str, Enum): SEQUENTIAL: str = "sequential" - RETRIEVAL: str = "retrieval" + RETRIEVAL: str = "retrieval" # TODO (travisdriver): change to DESCRIPTOR-BASED EXHAUSTIVE: str = "exhaustive" SEQUENTIAL_WITH_RETRIEVAL: str = "sequential_with_retrieval" RIG_HILTI: str = "rig_hilti" @@ -29,7 +29,7 @@ class RetrieverBase(GTSFMProcess): def __init__(self, matching_regime: ImageMatchingRegime) -> None: """ Args: - matching_regime: identifies type of matching used for image retrieval. + matching_regime: identifies type of matching used for image retrieval, e.g., exhaustive, descriptor-based. """ self._matching_regime = matching_regime