Reproducing all numbers in Tab 3

[vishaal27]

Hey, thanks for your great work.

I am able to reproduce the CLIP-Score and ImageReward numbers for the Preference Acc. column of tab. 3 of your paper using the test.py script. However, there is no script to run the evaluations for the other columns i.e. Recall and Filter. Could you please advise on how to reproduce these numbers as well? I think the data for reproducing that is not released yet right?

Thanks!

[xujz18]

The test set seems to be lost when we changed our server. I will release it if we find the test set. If necessary, you may try sampling from test set of ImageRewardDB yourself.