why the vit pretrained model not work well in inference ?

[HuangChiEn]

Although i haven't measure the accuracy but the prediction output seems seldom match with the correct label.

It must have been some steps I took it wrong, but not sure which to conduct the simple inference on the subset of ImageNet.
colab:https://colab.research.google.com/drive/1XD8yCv4IDBehceQ3onkW2pmIh6H4iq5f#scrollTo=o7VJxk3eUUoC

I have try to create an pretrained model vit_base_patch8_224 and make the corresponding torch transformation by using resolve_data_config and create_transform.

In colab script, the imagenette2 dataset is in used and I also have turn the model into eval mode.

Any suggestion is appreciated!!

[HuangChiEn]

my bad, imgnet cls to index mapping error..