Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Np add metadata.txt file to build indices #1435

Open
wants to merge 20 commits into
base: develop
Choose a base branch
from

Conversation

nikellepetrillo
Copy link
Contributor

@nikellepetrillo nikellepetrillo commented Nov 25, 2024

Description

This PR adds a task to the BuildIndices script that will build a text file with:

  • the pipeline-version
  • the file paths for the inputs references that are modified or used
  • the md5sums on all input files and output files
  • the date of workflow run (time stamp)
  • the workflow id

example .txt file looks like:

Date of Workflow Run: 2024-11-25T21:48:18Z

Input Files and their md5sums:
/cromwell_root/fc-e7f669fe-5a5c-4a8c-ad02-3b69105d0378/human/gencode.v43.primary_assembly.annotation.gtf : e00b8eb11ad5a2caaa252e0160ef350c
/cromwell_root/fc-e7f669fe-5a5c-4a8c-ad02-3b69105d0378/human/GRCh38.primary_assembly.genome.fa : 49bdb80d21a64dcb16acfc941843356e
/cromwell_root/fc-c40ec8a8-d60f-42f7-be36-3986b475190a/Biotypes.tsv : ad3206cda213d77fe39dc92bb02085c4

Output Files and their md5sums:
/cromwell_root/fc-e7f669fe-5a5c-4a8c-ad02-3b69105d0378/submissions/28f6a552-31b3-43d8-81a8-d409a7d87662/BuildIndices/e3bfb17d-9322-4b7f-b4d2-895cb011e10f/call-BuildStarSingleNucleus/cacheCopy/modified_star2.7.10a-Human-GENCODE-build-GRCh38-43.tar : 06fdf87249b42966973d8e4d9399ba40
/cromwell_root/fc-e7f669fe-5a5c-4a8c-ad02-3b69105d0378/submissions/28f6a552-31b3-43d8-81a8-d409a7d87662/BuildIndices/e3bfb17d-9322-4b7f-b4d2-895cb011e10f/call-BuildStarSingleNucleus/cacheCopy/modified_v43.annotation.gtf : 42d9352fcf93ff18cbe2aa074072f211
/cromwell_root/fc-e7f669fe-5a5c-4a8c-ad02-3b69105d0378/submissions/28f6a552-31b3-43d8-81a8-d409a7d87662/BuildIndices/e3bfb17d-9322-4b7f-b4d2-895cb011e10f/call-CalculateChromosomeSizes/cacheCopy/chrom.sizes : f94face851cb2129ab423f54db04fa0c
/cromwell_root/fc-e7f669fe-5a5c-4a8c-ad02-3b69105d0378/submissions/28f6a552-31b3-43d8-81a8-d409a7d87662/BuildIndices/e3bfb17d-9322-4b7f-b4d2-895cb011e10f/call-BuildBWAreference/cacheCopy/bwa-mem2-2.2.1-Human-GENCODE-build-GRCh38.tar : 3702fd2919568b5ddb03c57c535d4ee0

Workspace Bucket: fc-e7f669fe-5a5c-4a8c-ad02-3b69105d0378
Submission ID: 28f6a552-31b3-43d8-81a8-d409a7d87662
Workflow ID: e3bfb17d-9322-4b7f-b4d2-895cb011e10f

Checklist

If you can answer "yes" to the following items, please add a checkmark next to the appropriate checklist item(s) and notify our WARP team by tagging @broadinstitute/warp-admins in a comment on this PR.

  • Did you add inputs, outputs, or tasks to a workflow?
  • Did you modify, delete or move: file paths, file names, input names, output names, or task names?
  • If you made a changelog update, did you update the pipeline version number?

Copy link

Remember to squash merge!

Copy link

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
BuildIndices.wdl has not had its version updated
Some WDLs or changelog files need updating. See output for details.
validation_failed

Copy link

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
BuildIndices.changelog.md has not been changed and needs to be updated
Some changelog files need updating. See output for details.
validation_failed

Copy link

Remember to squash merge!

Copy link

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

Copy link

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
All WDLs and changelog files appear to be valid for this release.

@nikellepetrillo nikellepetrillo changed the title Np add mertadata.txt file to build indices Np add metadata.txt file to build indices Nov 26, 2024
Copy link
Contributor

@ekiernan ekiernan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much, Nikelle. This looks great!

@nikellepetrillo
Copy link
Contributor Author

Thank you so much, Nikelle. This looks great!

thanks liz! sorry, i accidentally requested you to re-review :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants