Add umicollapse as an alternative to umi-tools #1369

siddharthab · 2024-09-03T06:18:55Z

Checklist:

Benchmark paired ends mode
Include in multiqc - Add UMICollapse module MultiQC/MultiQC#2814
Decide if PREPARE_FOR_SALMON is needed
Update umicollapse version.
Update Changelog.

TODO: - [ ] Benchmark paired ends mode - [ ] Include in multiqc

MatthiasZepper

Excellent work, clean and well-structured! I think, you found very smart solutions to not duplicate too much of the code.

Of course, we should wait for the lead developers of the pipeline to review as well prior to merging, but in general you have my support for this addition! Ideally, your fix for umicollapse would also be included in the form of an updated tool version.

Please don't forget the Changelog later and also to add your name to the contributors (well deserved)!

nextflow_schema.json

MatthiasZepper · 2024-09-04T15:53:04Z

nextflow.config

@@ -30,6 +30,7 @@ params {
    with_umi                   = false
    skip_umi_extract           = false
    umitools_extract_method    = 'string'
+    umi_dedup_tool             = 'umicollapse'
    umitools_grouping_method   = 'directional'


Reading that makes we wonder if we should be conservative regarding the parameter names or make them more generic, e.g. renaming umitools_grouping_method to something like umi_grouping_method / umi_dedup_grouping_method etc.

Yes, I thought about that too. Depends on your policy for introducing breaking changes between versions. If the authors are OK, I can rename these parameters to be more generic.

workflows/rnaseq/main.nf

workflows/rnaseq/nextflow.config

siddharthab

Thanks for the quick review. I am tracking all remaining items in the PR summary.

siddharthab · 2024-09-04T16:37:49Z

nextflow.config

@@ -30,6 +30,7 @@ params {
    with_umi                   = false
    skip_umi_extract           = false
    umitools_extract_method    = 'string'
+    umi_dedup_tool             = 'umicollapse'
    umitools_grouping_method   = 'directional'


Yes, I thought about that too. Depends on your policy for introducing breaking changes between versions. If the authors are OK, I can rename these parameters to be more generic.

nextflow_schema.json

workflows/rnaseq/main.nf

workflows/rnaseq/nextflow.config

Signed-off-by: Siddhartha Bagaria <[email protected]>

siddharthab · 2024-09-04T22:20:05Z

I checked the output of UMICollapse, and it definitely does not follow the assumptions stated in prepare_for_rsem.py. So PREPARE_FOR_RSEM will be needed for UMICollapse as long as it is needed for umi-tools.

siddharthab · 2024-10-29T22:15:32Z

@MatthiasZepper all remaining tasks are now done. Please take another look.

MatthiasZepper · 2024-10-30T10:18:33Z

@MatthiasZepper all remaining tasks are now done. Please take another look.

Unfortunately, the UMICollapse module has not yet been upgraded...

MatthiasZepper · 2024-11-05T13:45:22Z

@siddharthab: The module is finally upgraded, so you can replace the local path with the up-to-date module.

siddharthab · 2024-11-05T22:59:40Z

@siddharthab: The module is finally upgraded, so you can replace the local path with the up-to-date module.

Thanks. Can you check if everything looks good now? I just ran nf-core modules update umicollapse, but do not know if I need to do more.

pinin4fjords

Can we uppercase the new process aliases please? My tiny little mind will get confused otherwise.

workflows/rnaseq/main.nf

Co-authored-by: Jonathan Manning <[email protected]>

siddharthab · 2024-11-06T21:31:41Z

Can we uppercase the new process aliases please?

@pinin4fjords, done.

MatthiasZepper

I can only reiterate that I would like to see this merged. In my opinion, an outstanding job.

pinin4fjords · 2024-11-12T18:31:14Z

@nf-core-bot fix linting

pinin4fjords · 2024-11-12T18:32:39Z

Sorry, good in principle now for me (maybe some factoring out we can do later). Just need to get tests to run + pass.

MatthiasZepper · 2024-11-13T15:56:58Z

Sorry, good in principle now for me (maybe some factoring out we can do later). Just need to get tests to run + pass.

Well, it appears the issue is on my end. Will fix asap! Sorry!

The UMICollapse tests are failing, because they try importing the BWA-Mem module, which isn’t included in this pipeline. As obviously everything is available in the modules directory, the tests pass in that context.

Considering that I had copied the tests from umitools dedup due to my limited experience with nf-test when creating the UMICollapse module, I was puzzled about why the umitools-tests work in here for the pipeline.

Turns out that I had coicidentially copied them as a template during a very brief period on March 4 before Maxime updated them again, which explains the discrepancy...

Long story short, I will have to fix the UMICollapse test in the modules directory to be self-reliant.

MatthiasZepper · 2024-11-24T16:47:39Z

Just to give a brief update: I have finally found some time to update the tests. Therefore, I hope the module tests will soon be updated in the repo and subsequently this PR also be merged successfully!

MatthiasZepper · 2024-12-02T11:03:23Z

@maxulysse: Let me know, when you have synced the test data to the S3 bucket. Ultimately, that sounds like something we could automate in the future with a CI on the test data repository?

maxulysse · 2024-12-04T13:49:17Z

Restore truncated CHANGELOG.md with 3.0 and prior releases.

🤦

I did not see that missing when I fixed the merge conflicts, good catch @MatthiasZepper

pinin4fjords · 2024-12-04T14:02:44Z

@nf-core-bot fix linting

MatthiasZepper · 2024-12-04T14:19:03Z

@nf-core-bot fix linting

Thanks! Strange, that the linter is unhappy, since I literally copy and pasted that from the current dev...anyway.

Add umicollapse as an alternative to umi-tools

fc13650

TODO: - [ ] Benchmark paired ends mode - [ ] Include in multiqc

siddharthab mentioned this pull request Sep 3, 2024

Add umicollapse and benchmark it against umitools dedup #1087

Open

MatthiasZepper approved these changes Sep 4, 2024

View reviewed changes

siddharthab commented Sep 4, 2024

View reviewed changes

Keep umitools as the default

761c56d

Signed-off-by: Siddhartha Bagaria <[email protected]>

siddharthab marked this pull request as ready for review September 4, 2024 16:46

siddharthab mentioned this pull request Sep 4, 2024

Option to keep chimeric reads after UMI deduplication #1373

Open

MatthiasZepper mentioned this pull request Oct 2, 2024

RC 3.16.0 #1395

Merged

11 tasks

siddharthab added 3 commits October 29, 2024 15:06

Merge branch 'dev' into umicollapse

acd8f65

update umicollapse to 1.1.0

e86e9a6

update CHANGELOG

28b10ac

actually update umicollapse

e5d9d4b

pinin4fjords reviewed Nov 6, 2024

View reviewed changes

Apply suggestions from code review

05513ee

Co-authored-by: Jonathan Manning <[email protected]>

siddharthab force-pushed the umicollapse branch from f4d6b0d to 05513ee Compare November 6, 2024 21:28

MatthiasZepper approved these changes Nov 12, 2024

View reviewed changes

Merge branch 'dev' into umicollapse

ee2dde2

[automated] Fix code linting

b372821

MatthiasZepper added 2 commits November 29, 2024 14:21

Update tests for UMICollapse module.

88d6411

Update umi-tools dedup tests at well.

15116a3

Update tests in subworkflows as well.

3bbd907

maxulysse and others added 2 commits December 4, 2024 13:54

Merge branch 'dev' into umicollapse

79d52f6

Restore truncated CHANGELOG.md with 3.0 and prior releases.

450b5b9

[automated] Fix code linting

a2dac87

MatthiasZepper merged commit 2986385 into nf-core:dev Dec 4, 2024
39 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add umicollapse as an alternative to umi-tools #1369

Add umicollapse as an alternative to umi-tools #1369

siddharthab commented Sep 3, 2024 •

edited

Loading

MatthiasZepper left a comment

MatthiasZepper Sep 4, 2024

siddharthab Sep 4, 2024

siddharthab left a comment

siddharthab Sep 4, 2024

siddharthab commented Sep 4, 2024

siddharthab commented Oct 29, 2024

MatthiasZepper commented Oct 30, 2024

MatthiasZepper commented Nov 5, 2024

siddharthab commented Nov 5, 2024

pinin4fjords left a comment •

edited

Loading

siddharthab commented Nov 6, 2024

MatthiasZepper left a comment

pinin4fjords commented Nov 12, 2024

pinin4fjords commented Nov 12, 2024

MatthiasZepper commented Nov 13, 2024 •

edited

Loading

MatthiasZepper commented Nov 24, 2024

MatthiasZepper commented Dec 2, 2024

maxulysse commented Dec 4, 2024

pinin4fjords commented Dec 4, 2024

MatthiasZepper commented Dec 4, 2024

Add umicollapse as an alternative to umi-tools #1369

Add umicollapse as an alternative to umi-tools #1369

Conversation

siddharthab commented Sep 3, 2024 • edited Loading

MatthiasZepper left a comment

Choose a reason for hiding this comment

MatthiasZepper Sep 4, 2024

Choose a reason for hiding this comment

siddharthab Sep 4, 2024

Choose a reason for hiding this comment

siddharthab left a comment

Choose a reason for hiding this comment

siddharthab Sep 4, 2024

Choose a reason for hiding this comment

siddharthab commented Sep 4, 2024

siddharthab commented Oct 29, 2024

MatthiasZepper commented Oct 30, 2024

MatthiasZepper commented Nov 5, 2024

siddharthab commented Nov 5, 2024

pinin4fjords left a comment • edited Loading

Choose a reason for hiding this comment

siddharthab commented Nov 6, 2024

MatthiasZepper left a comment

Choose a reason for hiding this comment

pinin4fjords commented Nov 12, 2024

pinin4fjords commented Nov 12, 2024

MatthiasZepper commented Nov 13, 2024 • edited Loading

MatthiasZepper commented Nov 24, 2024

MatthiasZepper commented Dec 2, 2024

maxulysse commented Dec 4, 2024

pinin4fjords commented Dec 4, 2024

MatthiasZepper commented Dec 4, 2024

siddharthab commented Sep 3, 2024 •

edited

Loading

pinin4fjords left a comment •

edited

Loading

MatthiasZepper commented Nov 13, 2024 •

edited

Loading