Run get_rep.py

[JackqqWang]

Hi Authors,

Thanks for your work.

When I tried to run get_rep.py, it seems we need to pass a parameter into “--config”. In your line 129 of file get_rep.py, you added a help saying "test config file path". However, I am sorry that I may not be able to get it.

For example, if we run: python get_rep.py --config ???   

Could you please provide a running script like the above one?

Thanks

[Adamdad]

Dear @JackqqWang,

Thank you for your interest. The get_rep.py script typically processes the configs for pre-trained models, computes the feature embeddings, and saves them as a .pth file.

You can find all the configurations for my experiments at https://github.com/Adamdad/DeRy/tree/main/configs/compute_sim. If you require additional configs, feel free to create your own or import pre-trained models from other repositories.

Best regards.

[Aris-z]

Hi @Adamdad,
I ran:

PYTHONPATH="$PWD" python simlarity/get_rep.py ./configs/compute_sim/resnet50_imagenet.py --out ./Feature/

but it raised error:

FileNotFoundError: [Errno 2] No such file or directory: 'data/imagenet/meta/val.txt'

and

FileNotFoundError: ImageNet: [Errno 2] No such file or directory: 'data/imagenet/meta/val.txt'

Did I run the code using the wrong config or the wrong way? Or it was the mistakenly use of the dataset?

Thanks.

[Adamdad]

Dear @Aris-z,

Thank you so much for expressing interest in our work.

It seems that you are encountering an ImageNet version issue. In older versions of ImageNet, data split files for train.txt and val.txt were provided, which the mmclassification package follows:
https://mmclassification.readthedocs.io/en/latest/tutorials/new_dataset.html

However, in some of the latest versions, the data split is no longer specified using two txt files, but instead by employing a list of folders with class names.

To resolve this issue, you can simply set the appropriate configuration to none in the config, which should then load the data according to the folder names.

ann_file=None

I hope this helps! If you have any further questions or concerns, please feel free to reach out.

Best regards.

[iiUMR]

Hi @Adamdad,
Thanks for your work. I encountered the same error as the one above when running get_rep.py, and then I modified the configuration file according to your answer: ann_file=None in configs/base/datasets/imagenet_bs64.py, but the following error was reported:
FileNotFoundError: [Errno 2] No such file or directory: 'data/imagenet/val' and FileNotFoundError: ImageNet: [Errno 2] No such file or directory: 'data/imagenet/val'.
Is this because I made a mistake in modifying the configuration file or is there another problem? I also hope you can give me answers.
Thanks.

[Adamdad]

Dear @xuelexuebuhui
I believe that is the problem associate with mmcls data loader. Please see you imagenet dataset folder and make sure the subfolder of data/imagenet/val is indeed there.

Best

[iiUMR]

indeed so! Thank you very much for clearing up my confusion.
But I don't know if I need to download this data set myself? If I need to download it myself, which folder should I put it in? And do I need to make changes to it? If there are other data sets that need to be downloaded, is there a way to download them all at once?
Sorry to bother you again.

[Adamdad]

Dear @xuelexuebuhui ,
We mainly use imagenet to extract feature and comparing models. You need to download it yourself. There are a lot of guideline/tutorial for downloading this very well-known dataset.

Best