Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release 1.2 #84

Merged
merged 80 commits into from
Dec 19, 2023
Merged

Release 1.2 #84

merged 80 commits into from
Dec 19, 2023

Commits on Mar 16, 2023

  1. Merge pull request #61 from sanger-tol/main 

    Syncing main to dev
    @priyanka-surana
    priyanka-surana authored Mar 16, 2023
    Configuration menu
    Copy the full SHA
    cf7b3e0 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2023

  1. add read group tag back

    @priyanka-surana
    priyanka-surana committed Apr 23, 2023
    Configuration menu
    Copy the full SHA
    c369d10 View commit details
    Browse the repository at this point in the history

  2. add crumble for pacbio hifi

    @priyanka-surana
    priyanka-surana committed Apr 23, 2023
    Configuration menu
    Copy the full SHA
    fd15eae View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2023

  1. update merge args

    @priyanka-surana
    priyanka-surana committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    45d4dec View commit details
    Browse the repository at this point in the history

  2. update bed output name

    @priyanka-surana
    priyanka-surana committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    0995917 View commit details
    Browse the repository at this point in the history

  3. update crumble outdir name

    @priyanka-surana
    priyanka-surana committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    a2a3751 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2023

  1. modules update

    @priyanka-surana
    priyanka-surana committed Jul 1, 2023
    Configuration menu
    Copy the full SHA
    188e708 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. prettier fix

    @priyanka-surana
    priyanka-surana committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    76e3c41 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2023

  1. Added the cleanup profile and use it for our Tower Sanger tests

    @muffato
    muffato committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    ff6fb57 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2023

  1. Latest nomenclature

    @muffato
    muffato committed Aug 13, 2023
    Configuration menu
    Copy the full SHA
    7fa1b83 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Use the latest version of the Tower action

    @muffato
    muffato committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    4b89961 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2023

  1. Our latest convention is to use one directory per pipeline, all under… 

    … `pipeline_info`
    @muffato
    muffato committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    0491dcc View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2023

  1. Update sanger_test.yml

    @gq1
    gq1 authored Aug 16, 2023
    Configuration menu
    Copy the full SHA
    a89dfee View commit details
    Browse the repository at this point in the history

  2. Update sanger_test_full.yml

    @gq1
    gq1 authored Aug 16, 2023
    Configuration menu
    Copy the full SHA
    1c724f6 View commit details
    Browse the repository at this point in the history

  3. Update sanger_test.yml

    @gq1
    gq1 authored Aug 16, 2023
    Configuration menu
    Copy the full SHA
    ab21cb1 View commit details
    Browse the repository at this point in the history

  4. Update sanger_test_full.yml

    @gq1
    gq1 authored Aug 16, 2023
    Configuration menu
    Copy the full SHA
    bc90069 View commit details
    Browse the repository at this point in the history

  5. Merge pull request #79 from sanger-tol/cleanup_ci 

    Added the cleanup profile and use it for our Tower Sanger tests
    @gq1
    gq1 authored Aug 16, 2023
    Configuration menu
    Copy the full SHA
    35119f4 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. New tolsoft details

    @muffato
    muffato committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    4d7bc41 View commit details
    Browse the repository at this point in the history

  2. Fixed the repository name

    @muffato
    muffato committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    aebfb83 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #80 from sanger-tol/new_tolsoft_details 

    New tolsoft details
    @gq1
    gq1 authored Aug 18, 2023
    Configuration menu
    Copy the full SHA
    093b313 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. Merge pull request #68 from sanger-tol/crumble 

    Add read group back and crumble for HiFi
    @muffato
    muffato authored Dec 6, 2023
    Configuration menu
    Copy the full SHA
    0262a3f View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2023

  1. Collect the read counts from the input files

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    2109bed View commit details
    Browse the repository at this point in the history

  2. Collect the size of the genome (file size is a good proxy)

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    e2e8ead View commit details
    Browse the repository at this point in the history

  3. Simply use the whole base name to name the channel

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    11678fc View commit details
    Browse the repository at this point in the history

  4. Deal with genomes > 4 Gbp 

    Closes #81
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    84caa10 View commit details
    Browse the repository at this point in the history

  5. Update the read count for SAMTOOLS_MERGE

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    8dca0f3 View commit details
    Browse the repository at this point in the history

  6. Some values are large and need 64 bits

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    8c67daf View commit details
    Browse the repository at this point in the history

  7. Wrong information

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    e432d89 View commit details
    Browse the repository at this point in the history

  8. Updated the error codes to the latest template versions. Covers all L… 

    …SF exit codes
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    ae6942b View commit details
    Browse the repository at this point in the history

  9. Estimate the resource requirements based on the size of the inputs

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    46e9910 View commit details
    Browse the repository at this point in the history

  10. bugfix: minimap2 uses the decimal system and understands floating poi… 

    …nt values !
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    5f2d786 View commit details
    Browse the repository at this point in the history

  11. The output of SAMTOOLS_MERGE is sorted

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    35a646d View commit details
    Browse the repository at this point in the history

  12. Logically, SAMTOOLS_MERGE should happen in the calling sub-workflow

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    04b2660 View commit details
    Browse the repository at this point in the history

  13. Skip SAMTOOLS_MERGE if there is a single file

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    f9b2763 View commit details
    Browse the repository at this point in the history

  14. Explain why there is no SAMTOOLS_SORT

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    fb1cd6d View commit details
    Browse the repository at this point in the history

  15. Replaced the markduplicate workflow by a single module / bash pipeline

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    c00eee6 View commit details
    Browse the repository at this point in the history

  16. The runtime of samtools sort depends on the number of reads

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    5c39de5 View commit details
    Browse the repository at this point in the history

  17. Updated requirements for SAMTOOLS_SORMADUP

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    ff9ff03 View commit details
    Browse the repository at this point in the history

  18. The MINIMAP2_ALIGN includes SAMTOOLS_SORT. Need some extra memory rel… 

    …ated to the number of reads
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    d54d46f View commit details
    Browse the repository at this point in the history

  19. In my latest tests, it seems BWAMEM2_MEM memory usage is correlated w… 

    …ith the logarithm of the genome size
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    582bedd View commit details
    Browse the repository at this point in the history

  20. I don't need samtools cat

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    d884d47 View commit details
    Browse the repository at this point in the history

  21. Alignment is nice

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    a581ae1 View commit details
    Browse the repository at this point in the history

  22. Also use -I to decrease the memory requirement of MINIMAP2_ALIGN

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    d141493 View commit details
    Browse the repository at this point in the history

  23. 20 minutes seems a bit too close to the real usage. We may want a lar… 

    …ger buffer
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    613c4cb View commit details
    Browse the repository at this point in the history

  24. Don't increase the number of CPUs too high as there are diminishing r… 

    …eturns
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    bc844bc View commit details
    Browse the repository at this point in the history

  25. Some samtools commands have a fixed memory usage per thread, so inclu… 

    …de the CPU count in the memory requirement
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    ec695e6 View commit details
    Browse the repository at this point in the history

  26. There was supposed to be a +1 there, to provide the "* task.attempt" … 

    …mechanism
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    d8043a5 View commit details
    Browse the repository at this point in the history

  27. Introduced a helper method that clearly shows how the logarithm is mo… 

    …dified
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    647a748 View commit details
    Browse the repository at this point in the history

  28. Increasing the number of attempts to give more resilience

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    d0b305a View commit details
    Browse the repository at this point in the history

  29. New formula that is less greedy

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    2465d74 View commit details
    Browse the repository at this point in the history

  30. Adjusted resource requirements for SAMTOOLS_SORMADUP

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    a9ec952 View commit details
    Browse the repository at this point in the history

  31. Added a note about minimap2

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    a249a76 View commit details
    Browse the repository at this point in the history

  32. Added some credit

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    99d289d View commit details
    Browse the repository at this point in the history

  33. More consistent comment

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    9afa763 View commit details
    Browse the repository at this point in the history

  34. Tell it's a meta map

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    eae3261 View commit details
    Browse the repository at this point in the history

  35. Indentation should be a multiple of 4

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    f211fb4 View commit details
    Browse the repository at this point in the history

  36. SAMTOOLS_FLAGSTAT may take more time

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    5b70604 View commit details
    Browse the repository at this point in the history

  37. typo

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    0c5e4f9 View commit details
    Browse the repository at this point in the history

  38. Increased runtime, just in case

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    c27dd30 View commit details
    Browse the repository at this point in the history

  39. Updated runtime and memory requirements

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    a2924b0 View commit details
    Browse the repository at this point in the history

  40. Need these fields in the debug output

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    b2f22a2 View commit details
    Browse the repository at this point in the history

  41. Like in the genome note pipeline, use the work directory instead of /tmp 

    Had to bump this samtools to 1.17 as the option is not available in earlier versions.
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    cfac7c8 View commit details
    Browse the repository at this point in the history

  42. Updated the settings for SORMADUP

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    9da9f3f View commit details
    Browse the repository at this point in the history

  43. Updated the BWA_MEM memory requirement

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    5e15589 View commit details
    Browse the repository at this point in the history

  44. Increased the number of retries

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    bdbcbb4 View commit details
    Browse the repository at this point in the history

  45. The default memory settings work just fine and make things easier to … 

    …manipulate
    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    3905219 View commit details
    Browse the repository at this point in the history

  46. Usage is very close to the trend line. Smaller bins work fine

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    c15d923 View commit details
    Browse the repository at this point in the history

  47. quay.io/ is now the default

    @muffato
    muffato committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    4094724 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. Added optimised settings for crumble

    @muffato
    muffato committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    3ef0477 View commit details
    Browse the repository at this point in the history

  2. Need to fake REF_PATH to force crumble to use the Fasta file defined … 

    …in the UR field of the @sq headers
    @muffato
    muffato committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    f0c1ba4 View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2023

  1. There is a difference for ONT, which I assume would be there for CLR too

    @muffato
    muffato committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    79bcbbc View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. Updated the docs before release

    @muffato
    muffato committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    63dfaff View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. add release name

    @priyanka-surana
    priyanka-surana authored Dec 18, 2023
    Configuration menu
    Copy the full SHA
    dda2c2e View commit details
    Browse the repository at this point in the history

  2. original version of the png file

    @priyanka-surana
    priyanka-surana authored Dec 18, 2023
    Configuration menu
    Copy the full SHA
    01748ac View commit details
    Browse the repository at this point in the history

  3. Forgot to mention --use_work_dir_as_temp

    @muffato
    muffato committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    c8e5125 View commit details
    Browse the repository at this point in the history

  4. Rename readmapping v3.pptx to readmapping_v3.pptx

    @priyanka-surana
    priyanka-surana authored Dec 18, 2023
    Configuration menu
    Copy the full SHA
    d3a84da View commit details
    Browse the repository at this point in the history

  5. Rename readmapping v2.pptx to readmapping_v2.pptx

    @priyanka-surana
    priyanka-surana authored Dec 18, 2023
    Configuration menu
    Copy the full SHA
    c522d3d View commit details
    Browse the repository at this point in the history

  6. Merge pull request #83 from sanger-tol/doc 

    Updated the docs before release
    @muffato
    muffato authored Dec 18, 2023
    Configuration menu
    Copy the full SHA
    bd02f75 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #82 from sanger-tol/resource_optimisation 

    Resource optimisation
    @muffato
    muffato authored Dec 18, 2023
    Configuration menu
    Copy the full SHA
    0f4e2a1 View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2023

  1. We're doing it a day early 

    We're so good. We're professionals. We're the best. ToL-IT power !
    @muffato
    muffato committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    db5c8b9 View commit details
    Browse the repository at this point in the history