Add GPU-based tests #6953
Add GPU-based tests #6953
Conversation
mashehu
requested review from
gitanoqevaporelmundoentero,
adamrtalbot and
sateeshperi
|
I love that we can do that. I'm sad because it'll mean I have to redo #6286 |
| path: ["${{ fromJson(needs.nf-test-changes.outputs.paths) }}"] | ||
| profile: [conda, docker_self_hosted, singularity] | ||
| include: | ||
| - path: modules/nf-core/parabricks/applybqsr |
There was a problem hiding this comment.
Will this trigger tests for these three modules whenever if: ( needs.nf-test-changes.outputs.paths != '[]' ) is true.
|
Let's touch,
to see how it goes in each case. Cheers! |
tests/config/nextflow.config
Outdated
| podman.runOptions = "--runtime crun --platform linux/x86_64 --systemd=always" | ||
| } else if ("$PROFILE" == "gpu") { | ||
| docker.runOptions = '-u $(id -u):$(id -g) --gpus all' | ||
| apptainer.runOptions = '--nv' | ||
| singularity.runOptions = '--nv' |
There was a problem hiding this comment.
I'd prefer this stuff to go in the configuration of the individual tools, so it can function as a reference implementation for users. Adding it here feels too "secret sauce" that will be hard for other people to follow.
Plus we should use process.containerOptions for process specific stuff.
Or we could add label 'gpu' to achieve a similar thing.
process {
withLabel "gpu" {
process.containerOptions = "--gpus all"
}
}
There was a problem hiding this comment.
How do we handle the singularity/apptainer option --nv then?
|
For reference, the gpu profile worked for the parabricks tests in #6881 |
Co-authored-by: Adam Talbot <[email protected]>
|
waiting to get again some modules updated on main, to see if they pick up things correctly. |
# Conflicts: # tests/config/nextflow.config
|
happy for now. need to refine the gpu test selection logic, but good to have it in for now |
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
There was a problem hiding this comment.
@mashehu Which one is the right gpu profile? 🫠
There was a problem hiding this comment.
There's two gpu profiles in this config now
moved the nf-test logic into a separate actions.yml so I can reuse it between for non-gpu and gpu cases. Did some clean-up in the action while at it.