Web-Server Parameters

[sbpkyle]

Hi,

I've been noticing some differences in the confidence metrics ran locally versus on the web server (No MSA).

I was wondering what parameters (if any) were different in regards to the # of diffusions and # trunk recycles.

[arogozhnikov]

AFAIK we use same settings: 5 samples are generated and ranked.
NB: there were recent changes in metrics #60 , not yet deployed to web server

[conorfhayes]

Does this include the seed? I am noticing significant difference between the server and local model scores. I am comparing between the same example prompts provided in the repo. Server gives a ~0.9 score while local model gives ~0.39.

[arogozhnikov]

Check that you either compute both with MSA, or compute both without MSA. Don't want to make any statements about same/not same seeds because they aren't helpful - even slight changes in code/ torch/cudnn version may drive to changes.