Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DRAFT] TSPS-183 Beagle imputation hg38 wdl and associated support wdls #1333

Open
wants to merge 73 commits into
base: develop
Choose a base branch
from
Open

[DRAFT] TSPS-183 Beagle imputation hg38 wdl and associated support wdls #1333

wants to merge 73 commits into from

Commits on Feb 7, 2024

  1. Merge pull request #1198 from broadinstitute/staging 

    Staging -> master
    @nikellepetrillo
    nikellepetrillo authored Feb 7, 2024
    Configuration menu
    Copy the full SHA
    9f137a7 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #1200 from broadinstitute/develop 

    dev -> staging
    @nikellepetrillo
    nikellepetrillo authored Feb 7, 2024
    Configuration menu
    Copy the full SHA
    92a887e View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. Merge pull request #1202 from broadinstitute/staging 

    Staging -> master
    @nikellepetrillo
    nikellepetrillo authored Feb 8, 2024
    Configuration menu
    Copy the full SHA
    045b998 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. Merge pull request #1208 from broadinstitute/develop 

    dev -> staging
    @nikellepetrillo
    nikellepetrillo authored Feb 16, 2024
    Configuration menu
    Copy the full SHA
    664f959 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. Merge pull request #1209 from broadinstitute/staging 

    Staging -> master
    @nikellepetrillo
    nikellepetrillo authored Feb 20, 2024
    Configuration menu
    Copy the full SHA
    b8a32c7 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2024

  1. wip add beagle imputation stuff

    @mmorgantaylor
    mmorgantaylor committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    c65dc53 View commit details
    Browse the repository at this point in the history

  2. add 2 wdls to dockstore.yml

    @mmorgantaylor
    mmorgantaylor committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    2306187 View commit details
    Browse the repository at this point in the history

  3. fix docker gar url

    @mmorgantaylor
    mmorgantaylor committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    85e19de View commit details
    Browse the repository at this point in the history

  4. use the right path for jars

    @mmorgantaylor
    mmorgantaylor committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    7e2be90 View commit details
    Browse the repository at this point in the history

  5. wip on imputation wdl

    @mmorgantaylor
    mmorgantaylor committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    9c64eeb View commit details
    Browse the repository at this point in the history

  6. oops use correct jar

    @mmorgantaylor
    mmorgantaylor committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    39482c4 View commit details
    Browse the repository at this point in the history

  7. missing equals

    @mmorgantaylor
    mmorgantaylor committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    f175179 View commit details
    Browse the repository at this point in the history

  8. fix java call again

    @mmorgantaylor
    mmorgantaylor committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    648f938 View commit details
    Browse the repository at this point in the history

Commits on Feb 29, 2024

  1. fix java call

    @mmorgantaylor
    mmorgantaylor committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    14357bf View commit details
    Browse the repository at this point in the history

  2. oops match file names

    @mmorgantaylor
    mmorgantaylor committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    457339b View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2024

  1. update beagle jar to 01Mar24.d36

    @mmorgantaylor
    mmorgantaylor committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    3fc22a8 View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2024

  1. debug GatherVcfs

    @mmorgantaylor
    mmorgantaylor committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    435ba19 View commit details
    Browse the repository at this point in the history

  2. debug GatherVcfs 2

    @mmorgantaylor
    mmorgantaylor committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    b72281a View commit details
    Browse the repository at this point in the history

  3. try to resolve missing file issue

    @mmorgantaylor
    mmorgantaylor committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    fd04907 View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2024

  1. don't impute over padding

    @mmorgantaylor
    mmorgantaylor committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    bce4c9d View commit details
    Browse the repository at this point in the history

  2. make the index again

    @mmorgantaylor
    mmorgantaylor committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    64d307f View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2024

  1. supply vcf_index input to SelectVariantsByIds

    @mmorgantaylor
    mmorgantaylor committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    54b05d6 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. update Imputation wdl too

    @mmorgantaylor
    mmorgantaylor committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    f2efe2c View commit details
    Browse the repository at this point in the history

  2. newlines

    @mmorgantaylor
    mmorgantaylor committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    0d3c862 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2024

  1. update for hg38

    @mmorgantaylor
    mmorgantaylor committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    3757137 View commit details
    Browse the repository at this point in the history

  2. Revert "update for hg38" 

    This reverts commit 3757137.
    @mmorgantaylor
    mmorgantaylor committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    30c634a View commit details
    Browse the repository at this point in the history

  3. update for hg38

    @mmorgantaylor
    mmorgantaylor committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    e8bf6ce View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2024

  1. liftover wdl

    @mmorgantaylor
    mmorgantaylor committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    fa3bbd4 View commit details
    Browse the repository at this point in the history

  2. remove GCP-specific vm commands

    @mmorgantaylor
    mmorgantaylor committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    7af23b4 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. use gatk

    @mmorgantaylor
    mmorgantaylor committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    b5bf5f4 View commit details
    Browse the repository at this point in the history

  2. fix suffix and basename

    @mmorgantaylor
    mmorgantaylor committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    cfac921 View commit details
    Browse the repository at this point in the history

  3. fix more filenames

    @mmorgantaylor
    mmorgantaylor committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    a7008d0 View commit details
    Browse the repository at this point in the history

  4. remove missing contig stuff for now

    @mmorgantaylor
    mmorgantaylor committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    57e00dd View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. fix ref panel path

    @mmorgantaylor
    mmorgantaylor committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    ff217eb View commit details
    Browse the repository at this point in the history

  2. another chr fix

    @mmorgantaylor
    mmorgantaylor committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    e64d04d View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. warn on missign contig

    @mmorgantaylor
    mmorgantaylor committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    d580de6 View commit details
    Browse the repository at this point in the history

  2. do fail if missing contig

    @mmorgantaylor
    mmorgantaylor committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    926cec1 View commit details
    Browse the repository at this point in the history

  3. more mem

    @mmorgantaylor
    mmorgantaylor committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    aa8f346 View commit details
    Browse the repository at this point in the history

  4. troubleshooting wld

    @mmorgantaylor
    mmorgantaylor committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    91c4c84 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. fixed plink path

    @mmorgantaylor
    mmorgantaylor committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    a41c366 View commit details
    Browse the repository at this point in the history

  2. add select_first test

    @mmorgantaylor
    mmorgantaylor committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    eb52486 View commit details
    Browse the repository at this point in the history

  3. cleanup

    @mmorgantaylor
    mmorgantaylor committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    223e01c View commit details
    Browse the repository at this point in the history

  4. add if block to test

    @mmorgantaylor
    mmorgantaylor committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    2c99b89 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. create and use ref panel interval list

    @mmorgantaylor
    mmorgantaylor committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    86188b6 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. move interval list creation to ref panel wdl

    @mmorgantaylor
    mmorgantaylor committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    3c786e6 View commit details
    Browse the repository at this point in the history

  2. give default values for optional inputs, weird

    @mmorgantaylor
    mmorgantaylor committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    c862073 View commit details
    Browse the repository at this point in the history

  3. change CountVariants calls

    @mmorgantaylor
    mmorgantaylor committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    b44bf4f View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2024

  1. test

    @mmorgantaylor
    mmorgantaylor committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    a122df6 View commit details
    Browse the repository at this point in the history

  2. add output to test

    @mmorgantaylor
    mmorgantaylor committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    9c90bca View commit details
    Browse the repository at this point in the history

  3. next test

    @mmorgantaylor
    mmorgantaylor committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    c845f84 View commit details
    Browse the repository at this point in the history

  4. more test

    @mmorgantaylor
    mmorgantaylor committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    471b5db View commit details
    Browse the repository at this point in the history

  5. another test

    @mmorgantaylor
    mmorgantaylor committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    e0284c4 View commit details
    Browse the repository at this point in the history

  6. update real task

    @mmorgantaylor
    mmorgantaylor committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    cc86b6b View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2024

  1. TSPS-226 presplit and prechunk beagle inputs (#1272) 

    *pre splitting and prechunking beagle imputation inputs to lower log numbers and storage account egress

---------

Co-authored-by: Jose Soto <[email protected]>
    @jsotobroad
    jsotobroad and Jose Soto authored Apr 27, 2024
    Configuration menu
    Copy the full SHA
    3b37aee View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. TSPS-221 remove index input and add seed to make beagle tool determin… 

    …istic (#1285)

* remove multi sample vcf index workflow input and add it to the PreSplitVcf task.
add seed number so that beagle is always deterministic. add comment to cpu input for PhaseAndImputeBeagle task

* change output_callset_name to output_base_name and remove optional outputs

* change n_failed_chunks ticket to an int

---------

Co-authored-by: Jose Soto <[email protected]>
    @jsotobroad
    jsotobroad and Jose Soto authored May 20, 2024
    Configuration menu
    Copy the full SHA
    19c095e View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. rename workflow

    @mmorgantaylor
    mmorgantaylor committed May 21, 2024
    Configuration menu
    Copy the full SHA
    5c0c0aa View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. TSPS-241 Clean up beagle wdl (#1288) 

    * clean up wdl with stuff from TSPS-241

* try to make fail fast work with double nested scatters

---------

Co-authored-by: Jose Soto <[email protected]>
    @jsotobroad
    jsotobroad and Jose Soto authored Jun 3, 2024
    Configuration menu
    Copy the full SHA
    916a94c View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. add specific gatk_docker

    @mmorgantaylor
    mmorgantaylor committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    4f18bdd View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. TSPS-142 updates to help creating simulated reference panel and runni… 

    …ng imputation against it (#1296)

* add optional error count override for testing

* rename reference base prefix variable and make it more user friendly

---------

Co-authored-by: Jose Soto <[email protected]>
    @jsotobroad
    jsotobroad and Jose Soto authored Jun 10, 2024
    Configuration menu
    Copy the full SHA
    b404f9b View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. add maxRetries 2 to all imputation beagle tasks

    @mmorgantaylor
    mmorgantaylor committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    808d0cf View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. add prechunk wdl to dockstore

    @mmorgantaylor
    mmorgantaylor committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    6201343 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. use acr for default ubuntu image

    @mmorgantaylor
    mmorgantaylor committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    f4ca86e View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. add preemptible 3

    @mmorgantaylor
    mmorgantaylor committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    cec272e View commit details
    Browse the repository at this point in the history

  2. use acr gatk docker as default

    @mmorgantaylor
    mmorgantaylor committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    4d7ec88 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. don't use preemptibles on GatherVcfs

    @mmorgantaylor
    mmorgantaylor committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    8cb21b1 View commit details
    Browse the repository at this point in the history

  2. basename fix for imputation beagle ref panel generation (#1332) 

    * try auto specifying chr at end of basename

* both tasks

* add liftovervcfs to dockstore

* allow specifying max mem
    @mmorgantaylor
    mmorgantaylor authored Jul 15, 2024
    Configuration menu
    Copy the full SHA
    c57210c View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. TSPS-269 Speed up CountVariantsInChunksBeagle by using bedtools (#1335) 

    * try creating bed files

* try again

* try again again

* a different thing

* use bedtools and bed ref panel files

* oops update the correct task

* fix

* use the right freaking file name

* remove comment
    @mmorgantaylor
    mmorgantaylor authored Jul 19, 2024
    Configuration menu
    Copy the full SHA
    95c3941 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. update pipeline version to 0.0.2

    @mmorgantaylor
    mmorgantaylor committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    700fdec View commit details
    Browse the repository at this point in the history

  2. Updated pipeline_versions.txt with all pipeline version information

    @actions-user
    actions-user committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    fd0ffed View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. TSPS-293: Fix up streaming imputation beagle (#1347) 

    update ImputationBeagle
    @mmorgantaylor
    mmorgantaylor authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    c0754cc View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2024

  1. add array imputation quota consumed wdl (#1425) 

    * add array imputation quota consumed wdl

* add changelogs for imputation array related workflows

---------

Co-authored-by: Jose Soto <[email protected]>
    @jsotobroad
    jsotobroad and Jose Soto authored Nov 13, 2024
    Configuration menu
    Copy the full SHA
    f4f6e47 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. TSPS-239 get wdl running on 400k sample ref panel (#1373) 

    * changes to help beagle imputation wdl run on a 400k sample reference panel

---------

Co-authored-by: Jose Soto <[email protected]>

* remove create imputation ref panel beagle wdl and changelog

* PR feedback

---------

Co-authored-by: Jose Soto <[email protected]>
Co-authored-by: M. Morgan Aster <[email protected]>
    @jsotobroad @mmorgantaylor
    3 people authored Nov 21, 2024
    Configuration menu
    Copy the full SHA
    fe04d7b View commit details
    Browse the repository at this point in the history

  2. add set -e -o pipefail to all relevant imputation tasks (#1434) 

    Co-authored-by: Jose Soto <[email protected]>
    @jsotobroad
    jsotobroad and Jose Soto authored Nov 21, 2024
    Configuration menu
    Copy the full SHA
    e51628a View commit details
    Browse the repository at this point in the history