-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Final manuscript analyses and edits in targets
pipeline
#202
Conversation
…y with prev pipeline
Co-authored-by: Dylan H. Morris <[email protected]>
…/wastewater-informed-covid-forecasting into hot-fix-full-targets-eval
Quick question on the relative score for summary stats on improving vs worsening forecast performance, and then this is ready for review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks, @kaitejohnson!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Noticed a couple places the calculation is right but mean_rel_{wis, crps}
should be rel_mean_{wis, crps}
so we don't confuse ourselves in the future.
This PR is a catch-all for a bunch of additional analyses and changes made to the azure pipeline and targets pipeline to facilitate the final results and supplementary figures for the analysis. At a high level this includes:
Makes the 4 main figures with each of their own specific datasets for: real-time relative performance comparison (with and without ww), real-time absolute performance compared to Hub models submitted Feb-Mar 2024, retrospective relative performance comparison across 23-24 season, retrospective absolute performance compared to Hub models across 23-24 season