Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Final manuscript analyses and edits in targets pipeline #202

Merged
merged 112 commits into from
Dec 12, 2024

Conversation

kaitejohnson
Copy link
Collaborator

@kaitejohnson kaitejohnson commented Oct 31, 2024

This PR is a catch-all for a bunch of additional analyses and changes made to the azure pipeline and targets pipeline to facilitate the final results and supplementary figures for the analysis. At a high level this includes:

Makes the 4 main figures with each of their own specific datasets for: real-time relative performance comparison (with and without ww), real-time absolute performance compared to Hub models submitted Feb-Mar 2024, retrospective relative performance comparison across 23-24 season, retrospective absolute performance compared to Hub models across 23-24 season

@kaitejohnson
Copy link
Collaborator Author

kaitejohnson commented Dec 12, 2024

Quick question on the relative score for summary stats on improving vs worsening forecast performance, and then this is ready for review

@dylanhmorris

dylanhmorris
dylanhmorris previously approved these changes Dec 12, 2024
Copy link
Collaborator

@dylanhmorris dylanhmorris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks, @kaitejohnson!

wweval/R/ms_fig4.R Outdated Show resolved Hide resolved
Copy link
Collaborator

@dylanhmorris dylanhmorris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noticed a couple places the calculation is right but mean_rel_{wis, crps} should be rel_mean_{wis, crps} so we don't confuse ourselves in the future.

wweval/R/ms_fig4.R Outdated Show resolved Hide resolved
wweval/R/ms_fig4.R Outdated Show resolved Hide resolved
wweval/R/ms_fig4.R Outdated Show resolved Hide resolved
wweval/R/ms_fig4.R Outdated Show resolved Hide resolved
wweval/R/ms_fig4.R Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants