Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SHAPEIT5_PHASERARE Migrate to nf-test #7064

Closed
wants to merge 58 commits into from

Conversation

LouisLeNezet
Copy link
Contributor

@LouisLeNezet LouisLeNezet commented Nov 22, 2024

PR checklist

Migration to nf-test for SHAPEIT5_PHASERARE process.
Modified stub and update shapeit5 version to 5.1.1

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

@LouisLeNezet LouisLeNezet requested a review from a team as a code owner November 22, 2024 17:37
@sateeshperi
Copy link
Contributor

upto you if you want to move the config to the top

Schmytzi and others added 21 commits December 4, 2024 09:40
* chore: homer/groseq => homer_groseq

* feat(homer): Add uniqmap to findpeaks

* docs: Update groseq workflow meta.yml

* chore: Covert pytest-workflow with Seqera AI

* test: Remove random test

* test(homer): Add empty uniqmap

* test(homer): Add tags and fix script that we're looking for

* chore: Remove versions

* test(homer): Add maketagdirectory stub

* test(homer): Add a stub for pos2bed

* test(homer): Clean up Seqera AI mistakes

* test(homer): Remove old md5sums from pytest

* test: Add samples from tutorial

* test: Add a stub test

* chore(homer): Run pytest-migrate on makeucscfile

* test(homer): Update makeucscfile

* test(homer): Add makeucscfile stub

* test(homer): Update makeucscfile stub

* build(homer): Add optional dependancies and build Seqera containers

* test(homer): bedGraph isn't deterministic

* test: Add -single flag

Makes the test slow

* test: Sort bedGraph

* style: Make nf-core subworkflows lint happy

* style: Run nf-core subworkflows lint --fix

* test: Remove actual lines from bedGraph

* build: Specify version and sort packages

uh-oh, this changes the name, we need to check that the names are sorted

* style: Clean up nf-core lint

* style(homer): LSP format everything
* ci: Just lint all the subworkflows

Takes less than 2 seconds to run locally.

* style: LSP format

* chore: Initialize subworkflows that have no tests for linting

* chore: Remove preprocess_rnaseq

* chore: Remove initialise subworkflow

Pretty sure this lives at
`subworkflows/local/utils_nfcore_<name>_pipeline/main.nf` now?

* style: Fix module name in meta.yml

* ci: Switch nf-core modules subworkflows lint to pre-commit

* style: Run lsp format

To test that pre-commit get's triggered
* Initial addition. Works.

* Updated conda version

* Updated snapshot

* Update modules/nf-core/coptr/merge/meta.yml

Co-authored-by: Igor Trujnara <[email protected]>

* Update modules/nf-core/coptr/merge/meta.yml

Co-authored-by: Igor Trujnara <[email protected]>

---------

Co-authored-by: Igor Trujnara <[email protected]>
* remove soft-linking of fasta instead stage in dir

* add comments for fasta stageAs
…f-core#7035)

use more descriptive variable names than it

Co-authored-by: Simon Pearce <[email protected]>
* first

* test

* use pigz

* fix test

* Update test
* Add nf-test

* Migrate to nf-test

* No version

* Update nf-test

* Remove tag.yml

---------

Co-authored-by: LouisLeNezet <[email protected]>
Co-authored-by: Sateesh_Peri <[email protected]>
* added sourcepredict module

* Update nextflow.config

edited nextflow.config

* Update main.nf.test

Edited the test file

* added test data for new sourcepredict module

* edited sourcepredict files

* fixed numba and ete3

* added test data location to test_data.config

* changed paths to nf-core new test data

* fixed whitspaces

* fixed whitspaces

* fixed prettier

* Update modules/nf-core/sourcepredict/main.nf

Co-authored-by: Sateesh_Peri <[email protected]>

* Update modules/nf-core/sourcepredict/main.nf

Co-authored-by: Sateesh_Peri <[email protected]>

* Update modules/nf-core/sourcepredict/main.nf

Co-authored-by: Sateesh_Peri <[email protected]>

* Update modules/nf-core/sourcepredict/main.nf

Co-authored-by: Sateesh_Peri <[email protected]>

* Update modules/nf-core/sourcepredict/main.nf

Co-authored-by: Sateesh_Peri <[email protected]>

* Update modules/nf-core/sourcepredict/tests/main.nf.test

Co-authored-by: Felix Lenner <[email protected]>

* fixed commits after first PR

* changed output file name to prevent file collision names

---------

Co-authored-by: Meriam van Os [email protected] <[email protected]>
Co-authored-by: Sateesh_Peri <[email protected]>
Co-authored-by: Felix Lenner <[email protected]>
* Added four file params to clustalo/align

* Update meta.yml

* Update modules/nf-core/clustalo/align/tests/main.nf.test

Co-authored-by: Luisa Santus <[email protected]>

* Update modules/nf-core/clustalo/align/tests/main.nf.test

Co-authored-by: Luisa Santus <[email protected]>

* Reran tests

---------

Co-authored-by: Luisa Santus <[email protected]>
* Bump GUNC and add gunc/run test

* Fix meta, use input_dir

* Add snapshots

* Change strategy to preserve filenames

* Add gunc/mergecheckm test

* Update tags

* Fix test names

* Update snapshots

* Add stubs

* Add stub snaps

* Address review comments

* Set args in nextflow.config again

* Comment problematic tests

* Comment another test

---------

Co-authored-by: Edmund Miller <[email protected]>
* Added subworkflow fasta_gxf_busco_plot

* Update subworkflows/nf-core/fasta_gxf_busco_plot/main.nf

Co-authored-by: Fernando Duarte <[email protected]>

* Update subworkflows/nf-core/fasta_gxf_busco_plot/main.nf

Co-authored-by: Fernando Duarte <[email protected]>

* Update subworkflows/nf-core/fasta_gxf_busco_plot/main.nf

Co-authored-by: Fernando Duarte <[email protected]>

* Update subworkflows/nf-core/fasta_gxf_busco_plot/main.nf

Co-authored-by: Fernando Duarte <[email protected]>

* Fixed linting and updated snapshot

* Applied suggestions

---------

Co-authored-by: Fernando Duarte <[email protected]>
* init fasta_create_index_bismark_bwameth subworkflow

* add tests + snap

* Change name

* update snap

---------

Co-authored-by: Simon Pearce <[email protected]>
* update malt/build module

* use stageAs for input fasta files

* Try HTTPS rather than s3

* Trying manually restricting memory

* Move config loading to the right place

* Update modules/nf-core/malt/build/tests/main.nf.test

Co-authored-by: Mahesh Binzer-Panchal <[email protected]>

* Update main.nf.test

* Update nextflow.config

* Get tests running

* Adjust to VM minimum

* And MALT_RUN

* use process.resourceLimits defining the memory limit

* Update modules/nf-core/malt/run/tests/nextflow.config

Co-authored-by: Mahesh Binzer-Panchal <[email protected]>

* Update modules/nf-core/malt/run/tests/nextflow.config

Co-authored-by: Mahesh Binzer-Panchal <[email protected]>

* Update modules/nf-core/malt/run/tests/nextflow.config

Co-authored-by: Mahesh Binzer-Panchal <[email protected]>

* Update nextflow.config

---------

Co-authored-by: James A. Fellows Yates <[email protected]>
Co-authored-by: Mahesh Binzer-Panchal <[email protected]>
* move mafft to mafft_align

* add mafft guidetree

* update mafft align

* add mafft guidetree tests

* fixes

* up

* update other mafft tests

* update mafft

* fix

* don't lint deleted files

* filter out main.nf.test and main.nf.snapshot

* try different strategy

* fix other

* fix other

* add tests

* remove pytest

* up

* update nftest

* fix

---------

Co-authored-by: mashehu <[email protected]>
* remove pigz

* add fix line

* test
* update trinity version and snapshot

* use commas in file extension pattern

---------

Co-authored-by: Sateesh_Peri <[email protected]>
* Add module gzrt (nf-core#6918)

* fixed environment.yml

* Reply to review

* gzrecover and gzip oneliner

* gzrt uses a named pipe to avoid temp files

---------

Co-authored-by: tm4zza <[email protected]>
abartlett004 and others added 27 commits December 4, 2024 09:41
* spaceranger_bump_version

* build: Use ADD to download tar.gz

* style: Clean up Dockerfile

* chore: Update labels and use an ARG

* chore: Switch base image to match Seqera Containers

* build: Make a manual checksum

* chore: Update [email protected]

* docs: Update documentation

* build: Bump image location and tag

* build: Can't actually get run of the tar.gz unzip step

Because of the Spaceranger URL.

* test(spaceranger): Add stub for mkref

* test: Bump snapshot

* test: Update versions yaml snapshot

* docs: Use a placeholder to avoid confusion

Co-authored-by: sebotic <[email protected]>

---------

Co-authored-by: Edmund Miller <[email protected]>
Co-authored-by: Edmund Miller <[email protected]>
Co-authored-by: sebotic <[email protected]>
* update propd module

* it works, need to add test

* create empty file when no DE genes were found; nf-test for permutation setting works

* modified propd module to save pairwise results optionally, and save empty output when no DE genes were found

* added additional tests in propd module

* round digits for reproducibility matters

* updated the test snapshots for propd

* update meta

* add description to meta

* fix meta

* lint propr/propd

* updated container for propr/propd

* update test snapshots for propr/propd

* [propr/propd] update container and snapshots

* [propr/propd] fix bug when defining the test configs

* [propr/propd] fix bug: only round numeric columns

* [propr/propd] update test snapshots

* [propr/propd] update containers

* [propr/propd] update test snapshots

* [propr/propd] update singularity url
* Create module

* parse arguments on module R script

* update meta info

* update nf-test

* update snaps after upgrading versions file

* update snaps after upgrading versions file

* add stub test, meta components and update names

* remove comments from ymls

* change output

* add meta to output channels

* fix environment.yml linting

* update snaps

* replace Stub with stub to pass linting

* Update main.nf.test

* change normalise by normalize

* move moduleBinaries to test nextflow config

* update snapshot after renaming

---------

Co-authored-by: Anabella Trigila <[email protected]>
* initialize module

* update test

* Remove TO-DO from main.nf.test

* Remove comment from main.nf.test

* update test after meta component in output channels

* split channel into png and txt outputs

* update meta file

* move moduleBinaries to test nextflow config
…e tests (nf-core#7075)

* Update the umi-tools dedup tests to use the new test data with UMIs.

* Update umicollapse tests to use the new UMI test data.

* Switch to nf-bam plugin for output validation.

---------

Co-authored-by: Matthias Zepper <[email protected]>
Co-authored-by: Jonathan Manning <[email protected]>
* start work on muse/call

* correct command and add snap

* prettier

* exclude conda

* change to bioconda

* update snap

* add conda test

* update meta

* change process label

* add indices and update meta

* update containers and version extraction
remove unused code from utils_nfcore_pipeline swf
* init bwameth subworkflow

* update output channels

* bwameth single, paired-end default, skip_dedup tests

* add GPU tests

* separate GPU tests

* add subworkflow GPU test path to test.yml include

* add subworkflow GPU test path to test.yml include

* add subworkflow GPU test path to test.yml include

* use more descriptive collect variables than it

* rename file to be explicit

* separate profile exclusions

* add gpu tag

* rm old test.yml

* add gpu test path to gpu-tests.yml:ci

* Added log_level: DEBUG

* Setup CI for debug

* add more debug steps

* usman's fix for exclude tags

* Updated fail condition

* ci sync to master

* fix filtering by usman

* Removed --changed-since when tags are supplied

* Enabled more checks and added a non-gpu module

* Two fixes

* Now using paths

* Fixed paths

* Fixed typo

* Now pass all paths instead of matrix

* Added confirm pass, cleaned dispatch and removed DEBUG

* Renamed all passes to confirm-pass

* Now using adamrtalbot/detect-nf-test-changes

* small ch format fix

---------

Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: Usman Rashid <[email protected]>
Improve input condition

Co-authored-by: Sateesh_Peri <[email protected]>
* new module plink genome

* formatting changes

* formatting changes

* stub, stub test and formatting fixes

* stub added

* new snapshot includes stub test

---------

Co-authored-by: Sateesh_Peri <[email protected]>
* Lint fastqs before and during preprocessing

* Add lint_log to snaps

* put back file removed in error

* Update snap

* Fix tests

* Linting snapshots to md5

* Update subworkflows/nf-core/fastq_qc_trim_filter_setstrandedness/main.nf

Co-authored-by: Maxime U Garcia <[email protected]>

* Update swf meta.yml

---------

Co-authored-by: Maxime U Garcia <[email protected]>
* style: Run lsp format

* refactor: txt => tsv

Allows this to be output directly to nf-core/differentialabundance
* start work on muse/sump

* update to include tbi

* uodate snap

* update paths

* prettier

* exclude conda test

* change to bioconda

* update meta and snap

* add conda test

* update tests

* change ext args to new convention

* update containers and version extraction and add gzip

* add bgzip

* Update modules/nf-core/muse/sump/meta.yml

Co-authored-by: Simon Pearce <[email protected]>

* update test

* finalize bgzip and specify tests

* fix meta

---------

Co-authored-by: Simon Pearce <[email protected]>
* Started to add clustalo, but the module doesn't support profile alignment yet

* Add clustalo as aligner

* Explicitly declare iterator variable

* Update subworkflows/nf-core/fasta_newick_epang_gappa/tests/main.nf.test

Co-authored-by: Simon Pearce <[email protected]>

* Update subworkflows/nf-core/fasta_newick_epang_gappa/tests/main.nf.test

Co-authored-by: Simon Pearce <[email protected]>

* Update subworkflows/nf-core/fasta_newick_epang_gappa/tests/main.nf.test

Co-authored-by: Simon Pearce <[email protected]>

* Update mistakes in tests

* Remove two tests with unstable md5 sums

* Update subworkflows/nf-core/fasta_newick_epang_gappa/tests/main.nf.test

Co-authored-by: Simon Pearce <[email protected]>

* Revert last update to test

---------

Co-authored-by: Simon Pearce <[email protected]>
* Initial addition

* Updated snapshot, removed obsolete snapshots

* Update modules/nf-core/coptr/extract/tests/main.nf.test

Added stub

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/coptr/extract/main.nf

Added stub

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/coptr/extract/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Updated stub

* Updated snapshot

* Update modules/nf-core/coptr/extract/main.nf

Added .pkl extension

Co-authored-by: Famke Bäuerle <[email protected]>

* Updated snapshot

---------

Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: Famke Bäuerle <[email protected]>
* Minor fixes to linting in prepro

* Add stub to fq/lint
* Minor refactor on attachMultiqcReport function.

The attachMultiqcReport function was renamed as the function does not attach the multiqc report, but rather simply tests if the input object contains zero, one, or more multiqc reports.

The function was also refactored to be more resilient to the type of object passed as input.
The function can now not only take a collection of zero, one, or more more multiqc reports, but will also handle the case if the input object is a Path object.

* fix switch brackets

---------

Co-authored-by: Júlia Mir Pedrol <[email protected]>
* Migrate to nf-test

* Add options stub

* Update modules/nf-core/shapeit5/phasecommon/tests/main.nf.test

Co-authored-by: Sateesh_Peri <[email protected]>

* Update modules/nf-core/shapeit5/phasecommon/tests/main.nf.test

Co-authored-by: Sateesh_Peri <[email protected]>

* Update test datasets

---------

Co-authored-by: LouisLeNezet <[email protected]>
Co-authored-by: Sateesh_Peri <[email protected]>
* Initial addition

* Update modules/nf-core/coptr/estimate/main.nf

Co-authored-by: Igor Trujnara <[email protected]>

* Update modules/nf-core/coptr/estimate/meta.yml

Co-authored-by: Igor Trujnara <[email protected]>

* Update modules/nf-core/coptr/estimate/meta.yml

Co-authored-by: Igor Trujnara <[email protected]>

* Fixed levels

* Fixed levels issue, task.process was repeated twice

* Fixed levels issue, task.process was repeated twice

* Fixed levels issue, task.process was repeated twice

* Fixed levels issue, task.process was repeated twice

* Fixed levels issue, task.process was repeated twice

---------

Co-authored-by: Igor Trujnara <[email protected]>
* add cellbender_empty_droplet_removal subworkflow

* correct the path to cellbender and anndata nf-core modules

* edit description of the output

* edited test for subworkflow cellbender_empty_droplet_removal

* add test snapshot (sigularity) for cellbender_empty_droplot_removal
subworkflows

* edited meta.yml to pass nf-core subworkflows lint

* updated to remove ch_h5ad and just emit h5ad=ANNDATA_BARCODES.out.h5ad

* rename subworkflow name to
H5AD_REMOVEBACKGROUND_BARCODES_CELLBENDER_ANNDATA from  CELLBENDER_EMPTY_DROPLET_REMOVAL
* Update glimpse2

* Fix argument name

* Fix nf-test

* Improve glimpse2 concordance

---------

Co-authored-by: Sateesh_Peri <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.