The model and loaded state dict do not match exactly

[TheoNanu]

I am trying to run the object detector on a video using the video_demo.py script.

The architecture used is Mask R-CNN with the SWIN-T backbone. The config file used is this one https://github.com/SwinTransformer/Swin-Transformer-Object-Detection/blob/master/configs/swin/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco.py and the model file is this one https://github.com/SwinTransformer/storage/releases/download/v1.0.3/mask_rcnn_swin_tiny_patch4_window7_1x.pth

I get the following message in the console and the output video has no detections on it.

`load checkpoint from local path: mask_rcnn_swin_tiny_patch4_window7_1x.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: backbone.layers.0.blocks.0.norm1.weight, backbone.layers.0.blocks.0.norm1.bias, backbone.layers.0.blocks.0.attn.relative_position_bias_table, backbone.layers.0.blocks.0.attn.relative_position_index, backbone.layers.0.blocks.0.attn.qkv.weight, backbone.layers.0.blocks.0.attn.qkv.bias, backbone.layers.0.blocks.0.attn.proj.weight, backbone.layers.0.blocks.0.attn.proj.bias, backbone.layers.0.blocks.0.norm2.weight, backbone.layers.0.blocks.0.norm2.bias, backbone.layers.0.blocks.0.mlp.fc1.weight, backbone.layers.0.blocks.0.mlp.fc1.bias, backbone.layers.0.blocks.0.mlp.fc2.weight, backbone.layers.0.blocks.0.mlp.fc2.bias, backbone.layers.0.blocks.1.norm1.weight, backbone.layers.0.blocks.1.norm1.bias, backbone.layers.0.blocks.1.attn.relative_position_bias_table, backbone.layers.0.blocks.1.attn.relative_position_index, backbone.layers.0.blocks.1.attn.qkv.weight, backbone.layers.0.blocks.1.attn.qkv.bias, backbone.layers.0.blocks.1.attn.proj.weight, backbone.layers.0.blocks.1.attn.proj.bias, backbone.layers.0.blocks.1.norm2.weight, backbone.layers.0.blocks.1.norm2.bias, backbone.layers.0.blocks.1.mlp.fc1.weight, backbone.layers.0.blocks.1.mlp.fc1.bias, backbone.layers.0.blocks.1.mlp.fc2.weight, backbone.layers.0.blocks.1.mlp.fc2.bias, backbone.layers.0.downsample.reduction.weight, backbone.layers.0.downsample.norm.weight, backbone.layers.0.downsample.norm.bias, backbone.layers.1.blocks.0.norm1.weight, backbone.layers.1.blocks.0.norm1.bias, backbone.layers.1.blocks.0.attn.relative_position_bias_table, backbone.layers.1.blocks.0.attn.relative_position_index, backbone.layers.1.blocks.0.attn.qkv.weight, backbone.layers.1.blocks.0.attn.qkv.bias, backbone.layers.1.blocks.0.attn.proj.weight, backbone.layers.1.blocks.0.attn.proj.bias, backbone.layers.1.blocks.0.norm2.weight, backbone.layers.1.blocks.0.norm2.bias, backbone.layers.1.blocks.0.mlp.fc1.weight, backbone.layers.1.blocks.0.mlp.fc1.bias, backbone.layers.1.blocks.0.mlp.fc2.weight, backbone.layers.1.blocks.0.mlp.fc2.bias, backbone.layers.1.blocks.1.norm1.weight, backbone.layers.1.blocks.1.norm1.bias, backbone.layers.1.blocks.1.attn.relative_position_bias_table, backbone.layers.1.blocks.1.attn.relative_position_index, backbone.layers.1.blocks.1.attn.qkv.weight, backbone.layers.1.blocks.1.attn.qkv.bias, backbone.layers.1.blocks.1.attn.proj.weight, backbone.layers.1.blocks.1.attn.proj.bias, backbone.layers.1.blocks.1.norm2.weight, backbone.layers.1.blocks.1.norm2.bias, backbone.layers.1.blocks.1.mlp.fc1.weight, backbone.layers.1.blocks.1.mlp.fc1.bias, backbone.layers.1.blocks.1.mlp.fc2.weight, backbone.layers.1.blocks.1.mlp.fc2.bias, backbone.layers.1.downsample.reduction.weight, backbone.layers.1.downsample.norm.weight, backbone.layers.1.downsample.norm.bias, backbone.layers.2.blocks.0.norm1.weight, backbone.layers.2.blocks.0.norm1.bias, backbone.layers.2.blocks.0.attn.relative_position_bias_table, backbone.layers.2.blocks.0.attn.relative_position_index, backbone.layers.2.blocks.0.attn.qkv.weight, backbone.layers.2.blocks.0.attn.qkv.bias, backbone.layers.2.blocks.0.attn.proj.weight, backbone.layers.2.blocks.0.attn.proj.bias, backbone.layers.2.blocks.0.norm2.weight, backbone.layers.2.blocks.0.norm2.bias, backbone.layers.2.blocks.0.mlp.fc1.weight, backbone.layers.2.blocks.0.mlp.fc1.bias, backbone.layers.2.blocks.0.mlp.fc2.weight, backbone.layers.2.blocks.0.mlp.fc2.bias, backbone.layers.2.blocks.1.norm1.weight, backbone.layers.2.blocks.1.norm1.bias, backbone.layers.2.blocks.1.attn.relative_position_bias_table, backbone.layers.2.blocks.1.attn.relative_position_index, backbone.layers.2.blocks.1.attn.qkv.weight, backbone.layers.2.blocks.1.attn.qkv.bias, backbone.layers.2.blocks.1.attn.proj.weight, backbone.layers.2.blocks.1.attn.proj.bias, backbone.layers.2.blocks.1.norm2.weight, backbone.layers.2.blocks.1.norm2.bias, backbone.layers.2.blocks.1.mlp.fc1.weight, backbone.layers.2.blocks.1.mlp.fc1.bias, backbone.layers.2.blocks.1.mlp.fc2.weight, backbone.layers.2.blocks.1.mlp.fc2.bias, backbone.layers.2.blocks.2.norm1.weight, backbone.layers.2.blocks.2.norm1.bias, backbone.layers.2.blocks.2.attn.relative_position_bias_table, backbone.layers.2.blocks.2.attn.relative_position_index, backbone.layers.2.blocks.2.attn.qkv.weight, backbone.layers.2.blocks.2.attn.qkv.bias, backbone.layers.2.blocks.2.attn.proj.weight, backbone.layers.2.blocks.2.attn.proj.bias, backbone.layers.2.blocks.2.norm2.weight, backbone.layers.2.blocks.2.norm2.bias, backbone.layers.2.blocks.2.mlp.fc1.weight, backbone.layers.2.blocks.2.mlp.fc1.bias, backbone.layers.2.blocks.2.mlp.fc2.weight, backbone.layers.2.blocks.2.mlp.fc2.bias, backbone.layers.2.blocks.3.norm1.weight, backbone.layers.2.blocks.3.norm1.bias, backbone.layers.2.blocks.3.attn.relative_position_bias_table, backbone.layers.2.blocks.3.attn.relative_position_index, backbone.layers.2.blocks.3.attn.qkv.weight, backbone.layers.2.blocks.3.attn.qkv.bias, backbone.layers.2.blocks.3.attn.proj.weight, backbone.layers.2.blocks.3.attn.proj.bias, backbone.layers.2.blocks.3.norm2.weight, backbone.layers.2.blocks.3.norm2.bias, backbone.layers.2.blocks.3.mlp.fc1.weight, backbone.layers.2.blocks.3.mlp.fc1.bias, backbone.layers.2.blocks.3.mlp.fc2.weight, backbone.layers.2.blocks.3.mlp.fc2.bias, backbone.layers.2.blocks.4.norm1.weight, backbone.layers.2.blocks.4.norm1.bias, backbone.layers.2.blocks.4.attn.relative_position_bias_table, backbone.layers.2.blocks.4.attn.relative_position_index, backbone.layers.2.blocks.4.attn.qkv.weight, backbone.layers.2.blocks.4.attn.qkv.bias, backbone.layers.2.blocks.4.attn.proj.weight, backbone.layers.2.blocks.4.attn.proj.bias, backbone.layers.2.blocks.4.norm2.weight, backbone.layers.2.blocks.4.norm2.bias, backbone.layers.2.blocks.4.mlp.fc1.weight, backbone.layers.2.blocks.4.mlp.fc1.bias, backbone.layers.2.blocks.4.mlp.fc2.weight, backbone.layers.2.blocks.4.mlp.fc2.bias, backbone.layers.2.blocks.5.norm1.weight, backbone.layers.2.blocks.5.norm1.bias, backbone.layers.2.blocks.5.attn.relative_position_bias_table, backbone.layers.2.blocks.5.attn.relative_position_index, backbone.layers.2.blocks.5.attn.qkv.weight, backbone.layers.2.blocks.5.attn.qkv.bias, backbone.layers.2.blocks.5.attn.proj.weight, backbone.layers.2.blocks.5.attn.proj.bias, backbone.layers.2.blocks.5.norm2.weight, backbone.layers.2.blocks.5.norm2.bias, backbone.layers.2.blocks.5.mlp.fc1.weight, backbone.layers.2.blocks.5.mlp.fc1.bias, backbone.layers.2.blocks.5.mlp.fc2.weight, backbone.layers.2.blocks.5.mlp.fc2.bias, backbone.layers.2.downsample.reduction.weight, backbone.layers.2.downsample.norm.weight, backbone.layers.2.downsample.norm.bias, backbone.layers.3.blocks.0.norm1.weight, backbone.layers.3.blocks.0.norm1.bias, backbone.layers.3.blocks.0.attn.relative_position_bias_table, backbone.layers.3.blocks.0.attn.relative_position_index, backbone.layers.3.blocks.0.attn.qkv.weight, backbone.layers.3.blocks.0.attn.qkv.bias, backbone.layers.3.blocks.0.attn.proj.weight, backbone.layers.3.blocks.0.attn.proj.bias, backbone.layers.3.blocks.0.norm2.weight, backbone.layers.3.blocks.0.norm2.bias, backbone.layers.3.blocks.0.mlp.fc1.weight, backbone.layers.3.blocks.0.mlp.fc1.bias, backbone.layers.3.blocks.0.mlp.fc2.weight, backbone.layers.3.blocks.0.mlp.fc2.bias, backbone.layers.3.blocks.1.norm1.weight, backbone.layers.3.blocks.1.norm1.bias, backbone.layers.3.blocks.1.attn.relative_position_bias_table, backbone.layers.3.blocks.1.attn.relative_position_index, backbone.layers.3.blocks.1.attn.qkv.weight, backbone.layers.3.blocks.1.attn.qkv.bias, backbone.layers.3.blocks.1.attn.proj.weight, backbone.layers.3.blocks.1.attn.proj.bias, backbone.layers.3.blocks.1.norm2.weight, backbone.layers.3.blocks.1.norm2.bias, backbone.layers.3.blocks.1.mlp.fc1.weight, backbone.layers.3.blocks.1.mlp.fc1.bias, backbone.layers.3.blocks.1.mlp.fc2.weight, backbone.layers.3.blocks.1.mlp.fc2.bias, backbone.patch_embed.proj.weight, backbone.patch_embed.proj.bias

missing keys in source state_dict: backbone.patch_embed.projection.weight, backbone.patch_embed.projection.bias, backbone.stages.0.blocks.0.norm1.weight, backbone.stages.0.blocks.0.norm1.bias, backbone.stages.0.blocks.0.attn.w_msa.relative_position_bias_table, backbone.stages.0.blocks.0.attn.w_msa.relative_position_index, backbone.stages.0.blocks.0.attn.w_msa.qkv.weight, backbone.stages.0.blocks.0.attn.w_msa.qkv.bias, backbone.stages.0.blocks.0.attn.w_msa.proj.weight, backbone.stages.0.blocks.0.attn.w_msa.proj.bias, backbone.stages.0.blocks.0.norm2.weight, backbone.stages.0.blocks.0.norm2.bias, backbone.stages.0.blocks.0.ffn.layers.0.0.weight, backbone.stages.0.blocks.0.ffn.layers.0.0.bias, backbone.stages.0.blocks.0.ffn.layers.1.weight, backbone.stages.0.blocks.0.ffn.layers.1.bias, backbone.stages.0.blocks.1.norm1.weight, backbone.stages.0.blocks.1.norm1.bias, backbone.stages.0.blocks.1.attn.w_msa.relative_position_bias_table, backbone.stages.0.blocks.1.attn.w_msa.relative_position_index, backbone.stages.0.blocks.1.attn.w_msa.qkv.weight, backbone.stages.0.blocks.1.attn.w_msa.qkv.bias, backbone.stages.0.blocks.1.attn.w_msa.proj.weight, backbone.stages.0.blocks.1.attn.w_msa.proj.bias, backbone.stages.0.blocks.1.norm2.weight, backbone.stages.0.blocks.1.norm2.bias, backbone.stages.0.blocks.1.ffn.layers.0.0.weight, backbone.stages.0.blocks.1.ffn.layers.0.0.bias, backbone.stages.0.blocks.1.ffn.layers.1.weight, backbone.stages.0.blocks.1.ffn.layers.1.bias, backbone.stages.0.downsample.norm.weight, backbone.stages.0.downsample.norm.bias, backbone.stages.0.downsample.reduction.weight, backbone.stages.1.blocks.0.norm1.weight, backbone.stages.1.blocks.0.norm1.bias, backbone.stages.1.blocks.0.attn.w_msa.relative_position_bias_table, backbone.stages.1.blocks.0.attn.w_msa.relative_position_index, backbone.stages.1.blocks.0.attn.w_msa.qkv.weight, backbone.stages.1.blocks.0.attn.w_msa.qkv.bias, backbone.stages.1.blocks.0.attn.w_msa.proj.weight, backbone.stages.1.blocks.0.attn.w_msa.proj.bias, backbone.stages.1.blocks.0.norm2.weight, backbone.stages.1.blocks.0.norm2.bias, backbone.stages.1.blocks.0.ffn.layers.0.0.weight, backbone.stages.1.blocks.0.ffn.layers.0.0.bias, backbone.stages.1.blocks.0.ffn.layers.1.weight, backbone.stages.1.blocks.0.ffn.layers.1.bias, backbone.stages.1.blocks.1.norm1.weight, backbone.stages.1.blocks.1.norm1.bias, backbone.stages.1.blocks.1.attn.w_msa.relative_position_bias_table, backbone.stages.1.blocks.1.attn.w_msa.relative_position_index, backbone.stages.1.blocks.1.attn.w_msa.qkv.weight, backbone.stages.1.blocks.1.attn.w_msa.qkv.bias, backbone.stages.1.blocks.1.attn.w_msa.proj.weight, backbone.stages.1.blocks.1.attn.w_msa.proj.bias, backbone.stages.1.blocks.1.norm2.weight, backbone.stages.1.blocks.1.norm2.bias, backbone.stages.1.blocks.1.ffn.layers.0.0.weight, backbone.stages.1.blocks.1.ffn.layers.0.0.bias, backbone.stages.1.blocks.1.ffn.layers.1.weight, backbone.stages.1.blocks.1.ffn.layers.1.bias, backbone.stages.1.downsample.norm.weight, backbone.stages.1.downsample.norm.bias, backbone.stages.1.downsample.reduction.weight, backbone.stages.2.blocks.0.norm1.weight, backbone.stages.2.blocks.0.norm1.bias, backbone.stages.2.blocks.0.attn.w_msa.relative_position_bias_table, backbone.stages.2.blocks.0.attn.w_msa.relative_position_index, backbone.stages.2.blocks.0.attn.w_msa.qkv.weight, backbone.stages.2.blocks.0.attn.w_msa.qkv.bias, backbone.stages.2.blocks.0.attn.w_msa.proj.weight, backbone.stages.2.blocks.0.attn.w_msa.proj.bias, backbone.stages.2.blocks.0.norm2.weight, backbone.stages.2.blocks.0.norm2.bias, backbone.stages.2.blocks.0.ffn.layers.0.0.weight, backbone.stages.2.blocks.0.ffn.layers.0.0.bias, backbone.stages.2.blocks.0.ffn.layers.1.weight, backbone.stages.2.blocks.0.ffn.layers.1.bias, backbone.stages.2.blocks.1.norm1.weight, backbone.stages.2.blocks.1.norm1.bias, backbone.stages.2.blocks.1.attn.w_msa.relative_position_bias_table, backbone.stages.2.blocks.1.attn.w_msa.relative_position_index, backbone.stages.2.blocks.1.attn.w_msa.qkv.weight, backbone.stages.2.blocks.1.attn.w_msa.qkv.bias, backbone.stages.2.blocks.1.attn.w_msa.proj.weight, backbone.stages.2.blocks.1.attn.w_msa.proj.bias, backbone.stages.2.blocks.1.norm2.weight, backbone.stages.2.blocks.1.norm2.bias, backbone.stages.2.blocks.1.ffn.layers.0.0.weight, backbone.stages.2.blocks.1.ffn.layers.0.0.bias, backbone.stages.2.blocks.1.ffn.layers.1.weight, backbone.stages.2.blocks.1.ffn.layers.1.bias, backbone.stages.2.blocks.2.norm1.weight, backbone.stages.2.blocks.2.norm1.bias, backbone.stages.2.blocks.2.attn.w_msa.relative_position_bias_table, backbone.stages.2.blocks.2.attn.w_msa.relative_position_index, backbone.stages.2.blocks.2.attn.w_msa.qkv.weight, backbone.stages.2.blocks.2.attn.w_msa.qkv.bias, backbone.stages.2.blocks.2.attn.w_msa.proj.weight, backbone.stages.2.blocks.2.attn.w_msa.proj.bias, backbone.stages.2.blocks.2.norm2.weight, backbone.stages.2.blocks.2.norm2.bias, backbone.stages.2.blocks.2.ffn.layers.0.0.weight, backbone.stages.2.blocks.2.ffn.layers.0.0.bias, backbone.stages.2.blocks.2.ffn.layers.1.weight, backbone.stages.2.blocks.2.ffn.layers.1.bias, backbone.stages.2.blocks.3.norm1.weight, backbone.stages.2.blocks.3.norm1.bias, backbone.stages.2.blocks.3.attn.w_msa.relative_position_bias_table, backbone.stages.2.blocks.3.attn.w_msa.relative_position_index, backbone.stages.2.blocks.3.attn.w_msa.qkv.weight, backbone.stages.2.blocks.3.attn.w_msa.qkv.bias, backbone.stages.2.blocks.3.attn.w_msa.proj.weight, backbone.stages.2.blocks.3.attn.w_msa.proj.bias, backbone.stages.2.blocks.3.norm2.weight, backbone.stages.2.blocks.3.norm2.bias, backbone.stages.2.blocks.3.ffn.layers.0.0.weight, backbone.stages.2.blocks.3.ffn.layers.0.0.bias, backbone.stages.2.blocks.3.ffn.layers.1.weight, backbone.stages.2.blocks.3.ffn.layers.1.bias, backbone.stages.2.blocks.4.norm1.weight, backbone.stages.2.blocks.4.norm1.bias, backbone.stages.2.blocks.4.attn.w_msa.relative_position_bias_table, backbone.stages.2.blocks.4.attn.w_msa.relative_position_index, backbone.stages.2.blocks.4.attn.w_msa.qkv.weight, backbone.stages.2.blocks.4.attn.w_msa.qkv.bias, backbone.stages.2.blocks.4.attn.w_msa.proj.weight, backbone.stages.2.blocks.4.attn.w_msa.proj.bias, backbone.stages.2.blocks.4.norm2.weight, backbone.stages.2.blocks.4.norm2.bias, backbone.stages.2.blocks.4.ffn.layers.0.0.weight, backbone.stages.2.blocks.4.ffn.layers.0.0.bias, backbone.stages.2.blocks.4.ffn.layers.1.weight, backbone.stages.2.blocks.4.ffn.layers.1.bias, backbone.stages.2.blocks.5.norm1.weight, backbone.stages.2.blocks.5.norm1.bias, backbone.stages.2.blocks.5.attn.w_msa.relative_position_bias_table, backbone.stages.2.blocks.5.attn.w_msa.relative_position_index, backbone.stages.2.blocks.5.attn.w_msa.qkv.weight, backbone.stages.2.blocks.5.attn.w_msa.qkv.bias, backbone.stages.2.blocks.5.attn.w_msa.proj.weight, backbone.stages.2.blocks.5.attn.w_msa.proj.bias, backbone.stages.2.blocks.5.norm2.weight, backbone.stages.2.blocks.5.norm2.bias, backbone.stages.2.blocks.5.ffn.layers.0.0.weight, backbone.stages.2.blocks.5.ffn.layers.0.0.bias, backbone.stages.2.blocks.5.ffn.layers.1.weight, backbone.stages.2.blocks.5.ffn.layers.1.bias, backbone.stages.2.downsample.norm.weight, backbone.stages.2.downsample.norm.bias, backbone.stages.2.downsample.reduction.weight, backbone.stages.3.blocks.0.norm1.weight, backbone.stages.3.blocks.0.norm1.bias, backbone.stages.3.blocks.0.attn.w_msa.relative_position_bias_table, backbone.stages.3.blocks.0.attn.w_msa.relative_position_index, backbone.stages.3.blocks.0.attn.w_msa.qkv.weight, backbone.stages.3.blocks.0.attn.w_msa.qkv.bias, backbone.stages.3.blocks.0.attn.w_msa.proj.weight, backbone.stages.3.blocks.0.attn.w_msa.proj.bias, backbone.stages.3.blocks.0.norm2.weight, backbone.stages.3.blocks.0.norm2.bias, backbone.stages.3.blocks.0.ffn.layers.0.0.weight, backbone.stages.3.blocks.0.ffn.layers.0.0.bias, backbone.stages.3.blocks.0.ffn.layers.1.weight, backbone.stages.3.blocks.0.ffn.layers.1.bias, backbone.stages.3.blocks.1.norm1.weight, backbone.stages.3.blocks.1.norm1.bias, backbone.stages.3.blocks.1.attn.w_msa.relative_position_bias_table, backbone.stages.3.blocks.1.attn.w_msa.relative_position_index, backbone.stages.3.blocks.1.attn.w_msa.qkv.weight, backbone.stages.3.blocks.1.attn.w_msa.qkv.bias, backbone.stages.3.blocks.1.attn.w_msa.proj.weight, backbone.stages.3.blocks.1.attn.w_msa.proj.bias, backbone.stages.3.blocks.1.norm2.weight, backbone.stages.3.blocks.1.norm2.bias, backbone.stages.3.blocks.1.ffn.layers.0.0.weight, backbone.stages.3.blocks.1.ffn.layers.0.0.bias, backbone.stages.3.blocks.1.ffn.layers.1.weight, backbone.stages.3.blocks.1.ffn.layers.1.bias`

What should I do?

[AkashSrinivasulu]

hi @TheoNanu , I am also facing same issue. Were you able to resolve it?

Thanks