Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nextstrain-automation build-configs to phylogenetic worklfow #37

Merged
merged 2 commits into from
Apr 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions phylogenetic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,10 @@ Once you've run the build, you can view the results in auspice:

## Configuration

Configuration takes place entirely with the `Snakefile`. This can be read top-to-bottom, each rule
specifies its file inputs and output and also its parameters. There is little redirection and each
rule should be able to be reasoned with on its own.
Configuration for the workflow takes place entirely within the [defaults/config_dengue.ymal](defaults/config_dengue.yaml).
The analysis pipeline is contained in [Snakefile](Snakefile) with included [rules](rules).
Each rule specifies its file inputs and output and pulls its parameters from the config.
There is little redirection and each rule should be able to be reasoned with on its own.


### Using GenBank data
Expand Down Expand Up @@ -68,3 +69,17 @@ With access to AWS, this can be more quickly run as:
[auspice]: https://docs.nextstrain.org/projects/auspice/en/stable/index.html
[Installing Nextstrain guide]: https://docs.nextstrain.org/en/latest/install.html
[Running a Pathogen Workflow guide]: https://docs.nextstrain.org/en/latest/tutorials/running-a-workflow.html

### Deploying build

To run the workflow and automatically deploy the build to nextstrain.org,
you will need to have AWS credentials to run the following:

```
nextstrain build \
--env AWS_ACCESS_KEY_ID \
--env AWS_SECRET_ACCESS_KEY \
. \
deploy_all \
--configfile build-configs/nextstrain-automation/config.yaml
```
4 changes: 4 additions & 0 deletions phylogenetic/build-configs/nextstrain-automation/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
custom_rules:
- build-configs/nextstrain-automation/deploy.smk

deploy_url: "s3://nextstrain-data"
15 changes: 15 additions & 0 deletions phylogenetic/build-configs/nextstrain-automation/deploy.smk
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
"""
This part of the workflow handles automatic deployments of the zika build.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This part of the workflow handles automatic deployments of the zika build.
This part of the workflow handles automatic deployments of the dengue build.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@victorlin @joverlee521 , could we generalize this and just say "pathogen" build?

This way there's less to trip-over when applying these rules to other pathogens of interest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, submitted as a hotfix: b2c3655

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, +1 for less pathogen-specific stuff.

Uploads the build defined as the default output of the workflow through
the `all` rule from Snakefille
"""

rule deploy_all:
input: *rules.all.input
output: touch("results/deploy_all.done")
params:
deploy_url = config["deploy_url"]
shell:
"""
nextstrain remote upload {params.deploy_url} {input}
"""