Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Finish modernization #20

Merged
merged 20 commits into from
Nov 27, 2024
Merged

Finish modernization #20

merged 20 commits into from
Nov 27, 2024

Conversation

genehack
Copy link
Contributor

@genehack genehack commented Nov 16, 2024 •
edited
Loading

Description of proposed changes

Preview builds on staging:

This PR adds the following changes:

  • Pull Nextclade dataset during ingest; use that for genotype/clade assignment
  • Add upload action to ingest workflow
  • Convert nextclade and phylogenetic workflows to download data from S3
  • Sets an explicit, hard-coded clock rate of 2e-04±1e-05 for both genome and prM-E builds
  • Modify builds so that only genome gets run with --timetree
  • Fix authors versus abbreviated authors so that Auspice tips are labeled with abbreviated version
  • Adds frequencies panel to both builds
  • Filters out probable Clade I travel cases (country=China and dates around 2016-2017)
  • Adds in color.tsv generation based on what is in dengue repo, for better country-level chroma/geography matchup
  • Adds CI and ingest-to-phylo github actions

Related issue(s)

Checklist

  • Checks pass

@genehack
Download Nextclade dataset during ingest... [#2]
272d4c6 
...instead of depending on having one built locally.

Also corrects nextclade dataset name in the config.
@genehack
Add / tweak bits to allow upload of ingested files to S3 [#2]
5f61500
@genehack
Convert nextclade workflow to download inputs from S3 [#2]
12865fc
@genehack
Convert phylogenetic workflow to download inputs from S3 [#2]
52f3da5
@genehack
We have a gene-specific build so we don't need this comment [#2]
55daabf
@genehack
Set explicit clock rate of 2e-04 ± 1e-05 [#2]
967c4e7
@genehack
Only run timetree on genome build [#2]
8f9dc75
@genehack
Put params in order of use [#2]
5c62c7c
@genehack
Swap 'authors' -> 'full_authors' and 'abbr_authors' -> 'authors' [#2]
e953108 
This makes the metadata annotations in the Auspice visualization look
much nicer.
@genehack
Add frequencies panel [#2]
8bef86e 
This is _largely_ copy-pasta-ed from the `measles` repo; the one
significant change is I enabled the frequencies on both the whole
genome and prM-E builds, and I set the `min_date` param to
`2017-01-01`, because virtually all of the post-1927 samples are dated
2017 and later.
@genehack
Filter out Chinese travel cases from 2016-ish outbreak [#2]
0557dac
@genehack
Copypasta assign-colors.py and associated data from dengue [#2]
6625d78
@genehack
Remove old colors file; update config [#2]
84fea9b
@genehack genehack marked this pull request as ready for review November 21, 2024 22:13
joverlee521
joverlee521 requested changes Nov 25, 2024
View reviewed changes
Copy link

@joverlee521 joverlee521 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only reviewed the general workflows, leaving scientific review to others.

ingest/build-configs/ci/copy_example_data.smk Outdated Show resolved Hide resolved
.github/workflows/ci.yaml Show resolved Hide resolved
ingest/rules/nextclade.smk Show resolved Hide resolved
ingest/build-configs/ci/config.yaml Outdated Show resolved Hide resolved
phylogenetic/defaults/color_orderings.tsv Outdated Show resolved Hide resolved
@genehack genehack mentioned this pull request Nov 26, 2024
Open
@genehack
Update color_orderings for YFV [#2]
a1ec892 
Also small script tweak
@genehack
Build colors.tsv dynamically during phylo build {#2}
bc41b40
@genehack
Add CI workflow and configs for ingest and phylogeny workflows [#2]
bbd653e 
Also add custom rule import bits to phylogenetic/Snakefile
@genehack
Add example data for ingest and phylogenetic CI workflows [#2]
fd22561 
This is the full NCBI yellow-fever dataset (taxon ID 11089) as
downloaded on 21 Nov 2024, and the ingested sequences and metadata as
downloaded from Nextstrain on 21 Nov 2024.

CI _could_ work just fine without embedding this data in the repo, but
having a hard-coded example-data file means that we don't depend on
NCBI or Nextstrain.org for successful CI runs.
@genehack
Add ingest-to-phylo Github workflow [#2]
9d2aadb
@genehack
Update README [#2]
729a60f
@genehack
Clean up old cruft in nextclade workflow [#2]
04b92ca
@genehack genehack force-pushed the finish-modernization-2 branch from f1a35a6 to 04b92ca Compare November 26, 2024 02:25
@genehack
Copy link
Contributor Author

genehack commented Nov 26, 2024 •
edited
Loading

Updated in response to feedback — thanks @joverlee521!

Rebuilt previews: genome, prM-E

@joverlee521 joverlee521 mentioned this pull request Nov 26, 2024
Open
1 task
@genehack genehack merged commit 0972a30 into main Nov 27, 2024
5 checks passed
@genehack genehack deleted the finish-modernization-2 branch November 27, 2024 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joverlee521 joverlee521 joverlee521 approved these changes

@huddlej huddlej Awaiting requested review from huddlej

@trvrb trvrb Awaiting requested review from trvrb

@jameshadfield jameshadfield Awaiting requested review from jameshadfield

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Modernize repo
2 participants
@genehack @joverlee521