-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Add --cores all
to rebuild-mpxv.yaml
#149
Conversation
This is required for the action to be compatible with snakemake version >5.11.0 which started to require the `--cores` option For more context see: - nextstrain/cli#272 -https://bedfordlab.slack.com/archives/C01LCTT7JNN/p1681324279363569?thread_ts=1681293140.124269&cid=C01LCTT7JNN
@@ -35,6 +35,7 @@ jobs: | |||
. \ | |||
envdir env.d snakemake notify_on_deploy \ | |||
--configfiles config/config_mpxv.yaml config/nextstrain_automation.yaml \ | |||
--cores all \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I don't think we should use --cores all
because of nextstrain/cli#175. (I mentioned this earlier, but I might not have explained it well then.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't use all
whenever there was an explicit number given. I didn't know what to pass otherwise when --cpus
wasn't passed.
What does nextstrain build
use as --cpus
by default? CLI --help-all
said None
? So if there's no machine up it won't run? Or will it be like 0.00001?
What would you put instead in case of no --cpus
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It uses the default as configured in AWS Batch. That will vary for different organizations, but for us, it's currently 4.
For this case above, I would explicitly pass --cpus 4
to nextstrain build
to make the AWS Batch default explicit (until nextstrain/cli#175 is fixed) and then also --cores 4
to snakemake
(until nextstrain/cli#272 is fixed, probably via first-class env var management support).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To get things moving quickly I will merge with all as I cannot see this ever causing real issues with the way we run workflows - snakemake may schedule too many jobs overall but that doesn't actually harm us.
We can easily replace all --cores all
instances later
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean, ok, I guess, but I don't see how --cores all
here is in any way better or quicker than --cpus 4
+ --cores 4
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To make what I'd said earlier very clear: if we run concurrent workflows with small numbers of requested CPUs, they are likely to get packed onto the same machine, where overscheduling jobs by Snakemake is likely to make them all slower. Slow enough to notice? Maybe, maybe not. I don't know, as I rarely pay attention to our workflow execution times.
…ws-batch --exec`
This is required for the action to be compatible with snakemake version >5.11.0
which started to require the
--cores
optionFor more context see:
[build]
--cpus
isn't passed to thru when using--exec
cli#272https://bedfordlab.slack.com/archives/C01LCTT7JNN/p1681324279363569?thread_ts=1681293140.124269&cid=C01LCTT7JNN
Checks pass: https://github.com/nextstrain/monkeypox/actions/runs/4682454769/jobs/8296331746
I added
--cores all
when no--cpus
was passed