Object Detection - Having difficulty when attempting to fine-tune model

[dakotalw]

I have been working on a project where I am trying to scan satellite imagery for presence of a specific object type (corrals). I have two files TIF images that I am using for training and validation, with associated AOIs and bounding boxes for each image in geoJSON format.

I believe I have done everything properly for attaching my AOIs and bounding boxes to original imagery as well as the creation of my test and validation datasets.

All of my code is listed below:

IMPORTS

class_config = ClassConfig(names=['corral'], colors=['red'])
class_config

#Load images

val_img_uri = '{validation_img}.TIF'
val_raster_source = RasterioSource(val_img_uri,allow_streaming=True)
print('Shape: ',val_raster_source.shape, ' Type:', val_raster_source.dtype)

train_img_uri = '{train_img}.TIF'
train_raster_source = RasterioSource(train_img_uri,allow_streaming=True)
print('Shape: ',train_raster_source.shape, ' Type:', train_raster_source.dtype)

#Load label geoJSONs (every entry in my label is a corral)

val_label_uri = '{validation_labels].geojson'

val_crs_transformer = RasterioCRSTransformer.from_uri(val_img_uri)
val_vector_source = GeoJSONVectorSource(
    val_label_uri, val_crs_transformer, ignore_crs_field=True,
    vector_transformers=[
        ClassInferenceTransformer(
            default_class_id=class_config.get_class_id('corral'))])

train_label_uri = '{train_labels}.geojson'

train_crs_transformer = RasterioCRSTransformer.from_uri(train_img_uri)
train_vector_source = GeoJSONVectorSource(
    train_label_uri, train_crs_transformer, ignore_crs_field=True,
    vector_transformers=[
        ClassInferenceTransformer(
            default_class_id=class_config.get_class_id('corral'))])

# Define label sources from label vectors

train_label_raster_source = ObjectDetectionLabelSource(
    train_vector_source,
    extent=train_raster_source.extent,
    ioa_thresh = 0.2,
    clip = False)

val_label_raster_source = ObjectDetectionLabelSource(
    val_vector_source,
    extent=val_raster_source.extent,
    ioa_thresh = 0.2,
    clip = False)

# Define Areas of Interest

train_aoi_source= GeoJSONVectorSource(
    uri='{training_aois}.geojson',
    crs_transformer=train_crs_transformer,
    ignore_crs_field=True)


val_aoi_source= GeoJSONVectorSource(
    uri='{validation_aois}.geojson',
    crs_transformer=val_crs_transformer,
    ignore_crs_field=True)

#Logic to extract aoi polygons

val_aois = []
for i in val_aoi_source.get_dataframe()['geometry']:
    val_aois.append(i)

train_aois = []
for i in train_aoi_source.get_dataframe()['geometry']:
    train_aois.append(i)


# Create training and validation scene, convert to GeoDataset
train_scene = Scene(
    id='train_scene',
    raster_source=train_raster_source,
    label_source=train_label_raster_source,
    aoi_polygons=train_aois)

val_scene = Scene(
    id='val_scene',
    raster_source=val_raster_source,
    label_source=val_label_raster_source,
    aoi_polygons=val_aois)

train_ds = ObjectDetectionSlidingWindowGeoDataset(
    scene = train_scene,
    size=200,
    stride=200,
    transform=A.Resize(256, 256)
)
val_ds = ObjectDetectionSlidingWindowGeoDataset(
    scene = val_scene,
    size=200,
    stride=200,
    transform=A.Resize(256, 256)
)
len(train_ds), len(val_ds)

#Initialize Model
deeplab = deeplabv3_resnet50(num_classes=len(class_config) + 1)

data_cfg = ObjectDetectionGeoDataConfig(
    class_names=class_config.names,
    class_colors=class_config.colors,
    # scene_dataset=train_ds,
    # train_sz_rel= .8, 
    num_workers=0, # increase to use multi-processing
)

solver_cfg = SolverConfig(
    batch_sz=3,
    lr=3e-2,
)

learner_cfg = ObjectDetectionLearnerConfig(data=data_cfg, solver=solver_cfg)

learner = ObjectDetectionLearner(
    cfg=learner_cfg,
    output_dir='./train-demo/',
    model=deeplab,
    train_ds=train_ds,
    valid_ds=val_ds
)
learner.log_data_stats()

When I attempt to actually run the model however, I am presented with the following error stack:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
 in 
----> 1 learner.train(epochs=1)

[/opt/anaconda3/lib/python3.8/site-packages/rastervision/pytorch_learner/learner.py](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/lib/python3.8/site-packages/rastervision/pytorch_learner/learner.py) in train(self, epochs)
   1263         for epoch in range(start_epoch, end_epoch):
   1264             log.info(f'epoch: {epoch}')
-> 1265             train_metrics = self.train_epoch(
   1266                 optimizer=self.opt, step_scheduler=self.step_scheduler)
   1267             if self.epoch_scheduler:

[/opt/anaconda3/lib/python3.8/site-packages/rastervision/pytorch_learner/learner.py](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/lib/python3.8/site-packages/rastervision/pytorch_learner/learner.py) in train_epoch(self, optimizer, step_scheduler)
   1186                 batch = (x, y)
   1187                 optimizer.zero_grad()
-> 1188                 output = self.train_step(batch, batch_ind)
   1189                 output['train_loss'].backward()
   1190                 optimizer.step()

[/opt/anaconda3/lib/python3.8/site-packages/rastervision/pytorch_learner/object_detection_learner.py](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/lib/python3.8/site-packages/rastervision/pytorch_learner/object_detection_learner.py) in train_step(self, batch, batch_ind)
     68     def train_step(self, batch, batch_ind):
     69         x, y = batch
---> 70         loss_dict = self.model(x, y)
     71         return {'train_loss': loss_dict['total_loss']}
     72 

[/opt/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py) in _call_impl(self, *input, **kwargs)
   1128         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130             return forward_call(*input, **kwargs)
   1131         # Do not call functions when jit is used
   1132         full_backward_hooks, non_full_backward_hooks = [], []

TypeError: forward() takes 2 positional arguments but 3 were given

As I have tried to track this down, I think that my error is with how I am either loading in the model or how I am setting up the model to run. I have tried a few different models and methods of loading, and this error keeps coming up.

For what its worth - I am running this in an .ipynb markdown file. (based on my understanding - since rastervision 0.20, this should not make a difference)

Any clarity is appreciated!

[AdeelH]

Looks like you are trying to use DeepLab, which is a semantic segmentation model, for object detection. Try using one of the object detection models in TorchVision e.g. FasterRCNN.

For what its worth - I am running this in an .ipynb markdown file. (based on my understanding - since rastervision 0.20, this should not make a difference)

That's right.

[dakotalw]

Thank you. When I try running this model, I get an error message that TypeError: 'BoxList' object is not subscriptable.

Should I be initializing the bounding boxes in any specific way? I am currently just loading the model in the same way as before, just declaring the number of classes.

rcnn = fasterrcnn_resnet50_fpn_v2(num_classes= 2)

[AdeelH]

My bad. I should have added that you also need to wrap the Faster RCNN model in a TorchVisionODAdapter which converts the data to the right format for the model.

You can do something like:

from rastervision.pytorch_learner.object_detection_utils import TorchVisionODAdapter

model = TorchVisionODAdapter(rcnn)

Alternatively, you can also just pass an ObjectDetectionModelConfig to ObjectDetectionLearnerConfig instead of passing in the model yourself and the model will be created automatically.

[dakotalw]

That worked. Thank you so much for all of your help!