RLE and polygon masks don't seem to play nicely together

[jerpint]

Hello, I am trying to train a model on fashionpedia dataset: https://github.com/cvdfoundation/fashionpedia#annotations

The dataset itself is in coco format, and has a mix of Polygons and RLE annotations. I've noticed that when trying to load the RLE annotations, the code crashed with the following error:

 $ python tools/train.py configs/fashionpedia/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py

Traceback (most recent call last):
  File "/Users/jeremypinto/mmdetection/tools/train.py", line 244, in <module>
    main()
  File "/Users/jeremypinto/mmdetection/tools/train.py", line 233, in main
    train_detector(
  File "/Users/jeremypinto/mmdetection/mmdet/apis/train.py", line 244, in train_detector
    runner.run(data_loaders, cfg.workflow)
  File "/Users/jeremypinto/opt/miniconda3/envs/fashionpedia/lib/python3.9/site-packages/mmcv/runner/epoch_based_runner.py", line 136, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/Users/jeremypinto/opt/miniconda3/envs/fashionpedia/lib/python3.9/site-packages/mmcv/runner/epoch_based_runner.py", line 49, in train
    for i, data_batch in enumerate(self.data_loader):
  File "/Users/jeremypinto/opt/miniconda3/envs/fashionpedia/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 628, in __next__
    data = self._next_data()
  File "/Users/jeremypinto/opt/miniconda3/envs/fashionpedia/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 671, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/Users/jeremypinto/opt/miniconda3/envs/fashionpedia/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/Users/jeremypinto/opt/miniconda3/envs/fashionpedia/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/Users/jeremypinto/mmdetection/mmdet/datasets/custom.py", line 220, in __getitem__
    data = self.prepare_train_img(idx)
  File "/Users/jeremypinto/mmdetection/mmdet/datasets/custom.py", line 243, in prepare_train_img
    return self.pipeline(results)
  File "/Users/jeremypinto/mmdetection/mmdet/datasets/pipelines/compose.py", line 41, in __call__
    data = t(data)
  File "/Users/jeremypinto/mmdetection/mmdet/datasets/pipelines/loading.py", line 398, in __call__
    results = self._load_masks(results)
  File "/Users/jeremypinto/mmdetection/mmdet/datasets/pipelines/loading.py", line 353, in _load_masks
    [self.process_polygons(polygons) for polygons in gt_masks], h,
  File "/Users/jeremypinto/mmdetection/mmdet/datasets/pipelines/loading.py", line 353, in <listcomp>
    [self.process_polygons(polygons) for polygons in gt_masks], h,
  File "/Users/jeremypinto/mmdetection/mmdet/datasets/pipelines/loading.py", line 330, in process_polygons
    if len(polygon) % 2 == 0 and len(polygon) >= 6:
TypeError: len() of unsized object

This seems to happen when it hits an annotation with the following "segmentation" key:

      "segmentation": {
        "size": [
          1024,
          682
        ],
        "counts": "jiZ9:fo00K`0E023L0O020N020N001M200000002NXOh00N1100N20lRd0KYm[O0J4M500I700D:20K51N104J208AYTU:"
      },

Here is the config I am using:

# The new config inherits a base config to highlight the necessary modification
_base_ = '../mask_rcnn/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_coco.py'

classes = ('shirt, blouse',
 'top, t-shirt, sweatshirt',
 'sweater',
 'cardigan',
 'jacket',
 'vest',
 'pants',
 'shorts',
 'skirt',
 'coat',
 'dress',
 'jumpsuit',
 'cape',
 'glasses',
 'hat',
 'headband, head covering, hair accessory',
 'tie',
 'glove',
 'watch',
 'belt',
 'leg warmer',
 'tights, stockings',
 'sock',
 'shoe',
 'bag, wallet',
 'scarf',
 'umbrella',
 'hood',
 'collar',
 'lapel',
 'epaulette',
 'sleeve',
 'pocket',
 'neckline',
 'buckle',
 'zipper',
 'applique',
 'bead',
 'bow',
 'flower',
 'fringe',
 'ribbon',
 'rivet',
 'ruffle',
 'sequin',
 'tassel'
 )

# We also need to change the num_classes in head to match the dataset's annotation
model = dict(
    roi_head=dict(
        bbox_head=dict(num_classes=len(classes)),
        mask_head=dict(num_classes=len(classes))
        )
    )

# Modify dataset related settings
dataset_type = 'COCODataset'

data = dict(
    workers_per_gpu=0,
    train=dict(
        img_prefix='data/fashionpedia/train/',
        classes=classes,
        ann_file='data/fashionpedia/instances_attributes_train2020.json'),
    val=dict(
        img_prefix='data/fashionpedia/test/',
        classes=classes,
        ann_file='data/fashionpedia/instances_attributes_val2020.json'),
    test=dict(
        img_prefix='data/fashionpedia/test/',
        classes=classes,
        ann_file='data/fashionpedia/instances_attributes_val2020.json'))

# We can use the pre-trained Mask RCNN model to obtain higher performance
load_from = 'checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth'

I am new to MMDetection and following along with this tutorial: https://mmdetection.readthedocs.io/en/stable/2_new_data_model.html

I appreciate any help

[jerpint]

So I can confirm the issue is that the original fashionpedia annotation has a mix of RLE and Polygons, and mmdet seems to supports either/or, but not both simultaneously, i.e annotations must be either all RLE, or all Polygons. The only solution I have found was to convert the RLE annotations to polygons. After conversion, the training seems to run fine.

Here is a notebook to convert RLE masks to polygons on fashionpedia: https://colab.research.google.com/drive/1BQKlXit_e8USsRJmlTSIvr59li3GbWaT?usp=sharing