inference results for 11.03.23

[bkotzen]

No description provided.

[bkotzen]

@martinjankowiak new inference results! had to subsample the GISAID metadata to get it to work in memory; hopefully there will be a more elegant solution later

[bkotzen]

@martinjankowiak updating with 11.17 as well!

[martinjankowiak]

@bkotzen if you're doing data subsampling can you note that in the corresponding pair of metadata.txt with a new entry? something like
Percent of raw data included in inference: 93%

EDIT: actually why are the numbers of regions/sequences so different between the two??

[bkotzen]

Hey @martinjankowiak yes I am doing some significant downsampling to meet compute restrictions. However, I'm finalizing a fix right now and I will re-run results for dates where down-sampling occurred. I should be able to impute some better results by EOW and that will resolve this issue!

[bkotzen]

Just added in some more recent results - to make sure everything works. Now that I know it works on current data I can fix 11.17.2023

[martinjankowiak]

@bkotzen how does the subsampling work? when you update the PR can you please make sure to:

• delete all irrelevant/stale files
• update metadata.txt everywhere where necessary to indicate to indicate that data was subsampled

[bkotzen]

The subsampling randomly selected a proportion of the samples in the GISAID metadata and ran PyR0 and BVAS only on those samples.

I have now made some tweaks to my compute infrastructure and the preprocessing code in order to handle the large number of sequences.

The timeline is:
11/03 started subsampling, now I'm re-doing this run
11/17 significant subsampling, but I just deleted the original results (5c68de7) and replaced them with non-subsampled results (e10667a)
11/28 by this point I had created a solution where I didn't need to subsample anymore so these results are fine

[martinjankowiak]

@bkotzen i see so we're waiting for you to replace the 11/03 run?

[bkotzen]

Yep! 11/03 is almost done - will put it here ASAP

[bkotzen]

Ok @martinjankowiak! Everything is all set now 💪🏼 😤

[martinjankowiak]

great thanks @bkotzen !