Pathogen repo build #37
Workflow file for this run
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # DO NOT EDIT - GENERATED | |
| # This workflow is intended to be called by workflows in our various pathogen | |
| # build repos. See workflow-templates/pathogen-repo-builds.yaml (a "starter" | |
| # workflow) in this repo for an example of what the caller workflow looks like. | |
| name: Pathogen repo build | |
| defaults: | |
| run: | |
| # This is the same as GitHub Action's `bash` keyword as of 20 June 2023: | |
| # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsshell | |
| # | |
| # Completely spelling it out here so that GitHub can't change it out from under us | |
| # and we don't have to refer to the docs to know the expected behavior. | |
| shell: bash --noprofile --norc -eo pipefail {0} | |
| on: | |
| workflow_call: | |
| inputs: | |
| repo: | |
| description: >- | |
| Repository name with owner (e.g. nextstrain/zika). Defaults to the repository of the caller workflow. | |
| type: string | |
| default: ${{ github.repository }} | |
| required: false | |
| runtime: | |
| description: >- | |
| Nextstrain runtime under which to run the build. Currently only supports docker, conda, and aws-batch. Defaults to "docker". | |
| The aws-batch runtime requires AWS credentials. These may come directly from secrets or indirectly from assuming a role via GitHub Actions' OIDC provider. | |
| The following secrets are used if present: | |
| - AWS_ACCESS_KEY_ID - AWS_SECRET_ACCESS_KEY | |
| They must be defined in the repo's Actions secrets and passed to this workflow with `secrets: inherit`. | |
| If no secrets are present, the GitHubActionsRoleNextstrainBatchJobs role is assumed (in both senses of the verb). | |
| type: string | |
| default: docker | |
| required: false | |
| run: | |
| description: >- | |
| The full `nextstrain build` command to run for the build. Defaults to `nextstrain build .` | |
| Use the runtime input to select the runtime for the build instead of the runtime selection options to ensure that the runtime is properly set up within the GitHub Action job. | |
| The pathogen repo is cloned to the top level of the working directory of the GitHub Action, so use `.` to point to the pathogen repo directory. | |
| If your build runs longer than the 6 hour limit for GitHub Action jobs, consider using the `--detach` flag for the aws-batch runtime. | |
| All environment variables provided via the env input and all secrets provided via `secrets: inherit` can be passed to the build runtime via the `--env` option. | |
| type: string | |
| default: nextstrain build . | |
| required: false | |
| env: | |
| description: >- | |
| Environment variables to set for this reusable workflow since environment variables in the caller workflow are not propagated to reusable workflows. This is expected to be a string containing YAML. | |
| This is easily produced, for example, by pretending you're writing normal nested YAML within a literal multi-line block scalar (introduced by "|"): | |
| with: | |
| env: | | |
| FOO: bar | |
| I_CANT_BELIEVE: "it's not YAML" | |
| would_you_believe: | | |
| it's | |
| not | |
| yaml | |
| Do not use for secrets! Instead, pass them via GitHub Action's dedicated secrets mechanism. | |
| type: string | |
| default: "" | |
| required: false | |
| artifact-name: | |
| description: >- | |
| Name to use for the build output artifact uploaded at end of the workflow. | |
| If you're invoking this workflow multiple times from the same calling workflow, you should set this. Otherwise, the default "build-outputs" is probably fine. | |
| type: string | |
| default: build-outputs | |
| required: false | |
| artifact-paths: | |
| description: >- | |
| List of paths to include in the build output artifact uploaded at the end of the workflow, as a string following the format of the `paths` input of the `actions/upload-artifact` action. For example: | |
| with: | |
| artifact-paths: | | |
| results/ | |
| auspice/ | |
| logs/ | |
| The default paths included in the artifact are: | |
| build.log | |
| auspice/ | |
| results/ | |
| benchmarks/ | |
| logs/ | |
| .snakemake/log/ | |
| The "build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log). | |
| This is not supported for builds on AWS Batch because the workflow detaches from the build. Please use the `nextstrain build` command locally to reattach to AWS Batch builds to download outputs. | |
| type: string | |
| required: false | |
| workflow_dispatch: | |
| inputs: | |
| runtime: | |
| description: >- | |
| Nextstrain runtime under which to run the build. Currently only supports docker, conda, and aws-batch. Defaults to "docker". | |
| The aws-batch runtime requires AWS credentials. These may come directly from secrets or indirectly from assuming a role via GitHub Actions' OIDC provider. | |
| The following secrets are used if present: | |
| - AWS_ACCESS_KEY_ID - AWS_SECRET_ACCESS_KEY | |
| They must be defined in the repo's Actions secrets and passed to this workflow with `secrets: inherit`. | |
| If no secrets are present, the GitHubActionsRoleNextstrainBatchJobs role is assumed (in both senses of the verb). | |
| type: string | |
| default: docker | |
| required: false | |
| run: | |
| description: >- | |
| The full `nextstrain build` command to run for the build. Defaults to `nextstrain build .` | |
| Use the runtime input to select the runtime for the build instead of the runtime selection options to ensure that the runtime is properly set up within the GitHub Action job. | |
| The pathogen repo is cloned to the top level of the working directory of the GitHub Action, so use `.` to point to the pathogen repo directory. | |
| If your build runs longer than the 6 hour limit for GitHub Action jobs, consider using the `--detach` flag for the aws-batch runtime. | |
| All environment variables provided via the env input and all secrets provided via `secrets: inherit` can be passed to the build runtime via the `--env` option. | |
| type: string | |
| default: nextstrain build . | |
| required: false | |
| env: | |
| description: >- | |
| Environment variables to set for this reusable workflow since environment variables in the caller workflow are not propagated to reusable workflows. This is expected to be a string containing YAML. | |
| This is easily produced, for example, by pretending you're writing normal nested YAML within a literal multi-line block scalar (introduced by "|"): | |
| with: | |
| env: | | |
| FOO: bar | |
| I_CANT_BELIEVE: "it's not YAML" | |
| would_you_believe: | | |
| it's | |
| not | |
| yaml | |
| Do not use for secrets! Instead, pass them via GitHub Action's dedicated secrets mechanism. | |
| type: string | |
| default: "" | |
| required: false | |
| artifact-name: | |
| description: >- | |
| Name to use for the build output artifact uploaded at end of the workflow. | |
| If you're invoking this workflow multiple times from the same calling workflow, you should set this. Otherwise, the default "build-outputs" is probably fine. | |
| type: string | |
| default: build-outputs | |
| required: false | |
| artifact-paths: | |
| description: >- | |
| List of paths to include in the build output artifact uploaded at the end of the workflow, as a string following the format of the `paths` input of the `actions/upload-artifact` action. For example: | |
| with: | |
| artifact-paths: | | |
| results/ | |
| auspice/ | |
| logs/ | |
| The default paths included in the artifact are: | |
| build.log | |
| auspice/ | |
| results/ | |
| benchmarks/ | |
| logs/ | |
| .snakemake/log/ | |
| The "build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log). | |
| This is not supported for builds on AWS Batch because the workflow detaches from the build. Please use the `nextstrain build` command locally to reattach to AWS Batch builds to download outputs. | |
| type: string | |
| required: false | |
| repo: | |
| description: >- | |
| Repository name with owner (e.g. nextstrain/zika). | |
| type: string | |
| default: "" | |
| required: true | |
| env: | |
| NEXTSTRAIN_GITHUB_DIR: .git/nextstrain/.github | |
| permissions: | |
| id-token: write | |
| jobs: | |
| workflow-context: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - id: workflow-context | |
| uses: nextstrain/.github/actions/workflow-context@master | |
| outputs: | |
| repository: ${{ steps.workflow-context.outputs.repository }} | |
| sha: ${{ steps.workflow-context.outputs.sha }} | |
| run-build: | |
| needs: workflow-context | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Checkout build repository | |
| uses: actions/checkout@v3 | |
| with: | |
| repository: ${{ inputs.repo }} | |
| - # Need to run this after the build repo is cloned so that cloning the | |
| # build repo does not overwrite the .git dir and remove the extra support files | |
| # that we need from nextstrain/.github repo | |
| name: Checkout ${{ needs.workflow-context.outputs.repository }} (sha ${{ needs.workflow-context.outputs.sha }}) | |
| uses: actions/checkout@v3 | |
| with: | |
| repository: ${{ needs.workflow-context.outputs.repository }} | |
| ref: ${{ needs.workflow-context.outputs.sha }} | |
| path: ${{ env.NEXTSTRAIN_GITHUB_DIR }} | |
| - if: inputs.env | |
| name: Set environment variables | |
| env: | |
| env: ${{ inputs.env }} | |
| run: > | |
| # shellcheck disable=SC2154 | |
| echo "$env" | "$NEXTSTRAIN_GITHUB_DIR"/bin/yaml-to-envvars | tee -a "$GITHUB_ENV" | |
| - name: Set secrets as environment variables | |
| env: | |
| secrets: ${{ toJson(secrets) }} | |
| run: > | |
| # shellcheck disable=SC2154 | |
| echo "$secrets" | jq 'del(.github_token)' | "$NEXTSTRAIN_GITHUB_DIR"/bin/json-to-envvars | tee -a "$GITHUB_ENV" | |
| - if: inputs.runtime == 'aws-batch' | |
| uses: aws-actions/configure-aws-credentials@v2 | |
| with: | |
| aws-region: us-east-1 | |
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | |
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | |
| role-to-assume: ${{ secrets.AWS_ACCESS_KEY_ID == '' && 'arn:aws:iam::827581582529:role/GitHubActionsRoleNextstrainBatchJobs' || '' }} | |
| # XXX TODO: <https://github.com/aws-actions/configure-aws-credentials#credential-lifetime> | |
| - name: Setup runtime ${{ inputs.runtime }} | |
| uses: ./.git/nextstrain/.github/actions/setup-nextstrain-cli | |
| with: | |
| cli-version: ">=7.1.0" | |
| runtime: ${{ inputs.runtime }} | |
| - name: Run build via ${{ inputs.runtime }} | |
| env: | |
| NEXTSTRAIN_BUILD_COMMAND: ${{ inputs.run }} | |
| run: | | |
| # shellcheck disable=SC2154 | |
| set -x | |
| eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee build.log | |
| - if: ${{ inputs.runtime == 'aws-batch' }} | |
| name: Get AWS Batch job id | |
| id: aws-batch | |
| run: | | |
| echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < build.log)" >> "$GITHUB_ENV" | |
| - if: env.AWS_BATCH_JOB_ID | |
| name: Generate AWS Batch summary | |
| run: | | |
| "$NEXTSTRAIN_GITHUB_DIR"/bin/interpolate-env < "$NEXTSTRAIN_GITHUB_DIR"/text-templates/attach-aws-batch.md \ | |
| > "$GITHUB_STEP_SUMMARY" | |
| - if: always() | |
| uses: actions/upload-artifact@v3 | |
| with: | |
| if-no-files-found: warn | |
| name: ${{ inputs.artifact-name }} | |
| path: | | |
| build.log | |
| auspice/ | |
| results/ | |
| benchmarks/ | |
| logs/ | |
| .snakemake/log/ | |
| ${{ inputs.artifact-paths }} | |
| outputs: | |
| AWS_BATCH_JOB_ID: ${{ env.AWS_BATCH_JOB_ID }} | |
| # Wait for up to 6 hours (the maximum/default GitHub Actions job timeout¹) | |
| # for the AWS Batch job to finish. | |
| # | |
| # ¹ <https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits> | |
| wait-1: | |
| # XXX FIXME: drop this | |
| timeout-minutes: 1 | |
| needs: [run-build, workflow-context] | |
| if: needs.run-build.outputs.AWS_BATCH_JOB_ID | |
| runs-on: ubuntu-latest | |
| steps: | |
| # Uses needs.workflow-context.outputs | |
| - # Need to run this after the build repo is cloned so that cloning the | |
| # build repo does not overwrite the .git dir and remove the extra support files | |
| # that we need from nextstrain/.github repo | |
| name: Checkout ${{ needs.workflow-context.outputs.repository }} (sha ${{ needs.workflow-context.outputs.sha }}) | |
| uses: actions/checkout@v3 | |
| with: | |
| repository: ${{ needs.workflow-context.outputs.repository }} | |
| ref: ${{ needs.workflow-context.outputs.sha }} | |
| path: ${{ env.NEXTSTRAIN_GITHUB_DIR }} | |
| - if: inputs.runtime == 'aws-batch' | |
| uses: aws-actions/configure-aws-credentials@v2 | |
| with: | |
| aws-region: us-east-1 | |
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | |
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | |
| role-to-assume: ${{ secrets.AWS_ACCESS_KEY_ID == '' && 'arn:aws:iam::827581582529:role/GitHubActionsRoleNextstrainBatchJobs' || '' }} | |
| # XXX TODO: <https://github.com/aws-actions/configure-aws-credentials#credential-lifetime> | |
| - name: Setup runtime ${{ inputs.runtime }} | |
| uses: ./.git/nextstrain/.github/actions/setup-nextstrain-cli | |
| with: | |
| cli-version: ">=7.1.0" | |
| runtime: ${{ inputs.runtime }} | |
| - id: attach | |
| name: Attach to AWS Batch job | |
| env: | |
| AWS_BATCH_JOB_ID: ${{ needs.run-build.outputs.AWS_BATCH_JOB_ID }} | |
| run: | | |
| # See <https://docs.github.com/en/actions/managing-workflow-runs/canceling-a-workflow#steps-github-takes-to-cancel-a-workflow-run> | |
| interrupt() { | |
| echo "Trapped SIGINT (job timed out/cancelled); detaching" >&2 | |
| # XXX FIXME: instead of converting SIGINT → SIGTSTP here, use --detach-on-interrupt in new release | |
| kill -SIGTSTP % | |
| wait % | |
| exit 0 | |
| } | |
| trap interrupt SIGINT | |
| nextstrain build \ | |
| --aws-batch \ | |
| --attach "$AWS_BATCH_JOB_ID" \ | |
| --no-download \ | |
| . \ | |
| & | |
| wait % | |
| # Allow the workflow to be considered successful even if this job errors | |
| # due to cancellation (timing out). Unfortunately, this doesn't | |
| # distinguish between error from cancellation and error from command | |
| # failure, so we work around that below. | |
| continue-on-error: true | |
| # Emit a "conclusion" output for the job that's based on the built-in | |
| # conclusion (success, failure, cancelled) of the "attach" step above. | |
| # This is the conclusion we care about for the job since the job's own | |
| # "conclusion" is masked/transformed by "continue-on-error: true" above. | |
| outputs: | |
| attach-step-conclusion: ${{ steps.attach.conclusion }} | |
| # Wait for up to another 6 hours (hours 6–12) if the preceding wait-N job | |
| # timed out while attached to the AWS Batch job. | |
| wait-2: | |
| # XXX FIXME: drop this | |
| timeout-minutes: 1 | |
| runs-on: ubuntu-latest | |
| steps: | |
| # Uses needs.workflow-context.outputs | |
| - # Need to run this after the build repo is cloned so that cloning the | |
| # build repo does not overwrite the .git dir and remove the extra support files | |
| # that we need from nextstrain/.github repo | |
| name: Checkout ${{ needs.workflow-context.outputs.repository }} (sha ${{ needs.workflow-context.outputs.sha }}) | |
| uses: actions/checkout@v3 | |
| with: | |
| repository: ${{ needs.workflow-context.outputs.repository }} | |
| ref: ${{ needs.workflow-context.outputs.sha }} | |
| path: ${{ env.NEXTSTRAIN_GITHUB_DIR }} | |
| - if: inputs.runtime == 'aws-batch' | |
| uses: aws-actions/configure-aws-credentials@v2 | |
| with: | |
| aws-region: us-east-1 | |
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | |
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | |
| role-to-assume: ${{ secrets.AWS_ACCESS_KEY_ID == '' && 'arn:aws:iam::827581582529:role/GitHubActionsRoleNextstrainBatchJobs' || '' }} | |
| # XXX TODO: <https://github.com/aws-actions/configure-aws-credentials#credential-lifetime> | |
| - name: Setup runtime ${{ inputs.runtime }} | |
| uses: ./.git/nextstrain/.github/actions/setup-nextstrain-cli | |
| with: | |
| cli-version: ">=7.1.0" | |
| runtime: ${{ inputs.runtime }} | |
| - id: attach | |
| name: Attach to AWS Batch job | |
| env: | |
| AWS_BATCH_JOB_ID: ${{ needs.run-build.outputs.AWS_BATCH_JOB_ID }} | |
| run: | | |
| # See <https://docs.github.com/en/actions/managing-workflow-runs/canceling-a-workflow#steps-github-takes-to-cancel-a-workflow-run> | |
| interrupt() { | |
| echo "Trapped SIGINT (job timed out/cancelled); detaching" >&2 | |
| # XXX FIXME: instead of converting SIGINT → SIGTSTP here, use --detach-on-interrupt in new release | |
| kill -SIGTSTP % | |
| wait % | |
| exit 0 | |
| } | |
| trap interrupt SIGINT | |
| nextstrain build \ | |
| --aws-batch \ | |
| --attach "$AWS_BATCH_JOB_ID" \ | |
| --no-download \ | |
| . \ | |
| & | |
| wait % | |
| # Allow the workflow to be considered successful even if this job errors | |
| # due to cancellation (timing out). Unfortunately, this doesn't | |
| # distinguish between error from cancellation and error from command | |
| # failure, so we work around that below. | |
| continue-on-error: true | |
| # Emit a "conclusion" output for the job that's based on the built-in | |
| # conclusion (success, failure, cancelled) of the "attach" step above. | |
| # This is the conclusion we care about for the job since the job's own | |
| # "conclusion" is masked/transformed by "continue-on-error: true" above. | |
| outputs: | |
| attach-step-conclusion: ${{ steps.attach.conclusion }} | |
| needs: [wait-1, run-build, workflow-context] | |
| if: needs.wait-1.outputs.attach-step-conclusion == 'cancelled' | |
| # 12–18 hours | |
| wait-3: | |
| # XXX FIXME: drop this | |
| timeout-minutes: 1 | |
| runs-on: ubuntu-latest | |
| steps: | |
| # Uses needs.workflow-context.outputs | |
| - # Need to run this after the build repo is cloned so that cloning the | |
| # build repo does not overwrite the .git dir and remove the extra support files | |
| # that we need from nextstrain/.github repo | |
| name: Checkout ${{ needs.workflow-context.outputs.repository }} (sha ${{ needs.workflow-context.outputs.sha }}) | |
| uses: actions/checkout@v3 | |
| with: | |
| repository: ${{ needs.workflow-context.outputs.repository }} | |
| ref: ${{ needs.workflow-context.outputs.sha }} | |
| path: ${{ env.NEXTSTRAIN_GITHUB_DIR }} | |
| - if: inputs.runtime == 'aws-batch' | |
| uses: aws-actions/configure-aws-credentials@v2 | |
| with: | |
| aws-region: us-east-1 | |
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | |
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | |
| role-to-assume: ${{ secrets.AWS_ACCESS_KEY_ID == '' && 'arn:aws:iam::827581582529:role/GitHubActionsRoleNextstrainBatchJobs' || '' }} | |
| # XXX TODO: <https://github.com/aws-actions/configure-aws-credentials#credential-lifetime> | |
| - name: Setup runtime ${{ inputs.runtime }} | |
| uses: ./.git/nextstrain/.github/actions/setup-nextstrain-cli | |
| with: | |
| cli-version: ">=7.1.0" | |
| runtime: ${{ inputs.runtime }} | |
| - id: attach | |
| name: Attach to AWS Batch job | |
| env: | |
| AWS_BATCH_JOB_ID: ${{ needs.run-build.outputs.AWS_BATCH_JOB_ID }} | |
| run: | | |
| # See <https://docs.github.com/en/actions/managing-workflow-runs/canceling-a-workflow#steps-github-takes-to-cancel-a-workflow-run> | |
| interrupt() { | |
| echo "Trapped SIGINT (job timed out/cancelled); detaching" >&2 | |
| # XXX FIXME: instead of converting SIGINT → SIGTSTP here, use --detach-on-interrupt in new release | |
| kill -SIGTSTP % | |
| wait % | |
| exit 0 | |
| } | |
| trap interrupt SIGINT | |
| nextstrain build \ | |
| --aws-batch \ | |
| --attach "$AWS_BATCH_JOB_ID" \ | |
| --no-download \ | |
| . \ | |
| & | |
| wait % | |
| # Allow the workflow to be considered successful even if this job errors | |
| # due to cancellation (timing out). Unfortunately, this doesn't | |
| # distinguish between error from cancellation and error from command | |
| # failure, so we work around that below. | |
| continue-on-error: true | |
| # Emit a "conclusion" output for the job that's based on the built-in | |
| # conclusion (success, failure, cancelled) of the "attach" step above. | |
| # This is the conclusion we care about for the job since the job's own | |
| # "conclusion" is masked/transformed by "continue-on-error: true" above. | |
| outputs: | |
| attach-step-conclusion: ${{ steps.attach.conclusion }} | |
| needs: [wait-2, run-build, workflow-context] | |
| if: needs.wait-2.outputs.attach-step-conclusion == 'cancelled' | |
| # 18–24 hours | |
| wait-4: | |
| # XXX FIXME: drop this | |
| timeout-minutes: 1 | |
| runs-on: ubuntu-latest | |
| steps: | |
| # Uses needs.workflow-context.outputs | |
| - # Need to run this after the build repo is cloned so that cloning the | |
| # build repo does not overwrite the .git dir and remove the extra support files | |
| # that we need from nextstrain/.github repo | |
| name: Checkout ${{ needs.workflow-context.outputs.repository }} (sha ${{ needs.workflow-context.outputs.sha }}) | |
| uses: actions/checkout@v3 | |
| with: | |
| repository: ${{ needs.workflow-context.outputs.repository }} | |
| ref: ${{ needs.workflow-context.outputs.sha }} | |
| path: ${{ env.NEXTSTRAIN_GITHUB_DIR }} | |
| - if: inputs.runtime == 'aws-batch' | |
| uses: aws-actions/configure-aws-credentials@v2 | |
| with: | |
| aws-region: us-east-1 | |
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | |
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | |
| role-to-assume: ${{ secrets.AWS_ACCESS_KEY_ID == '' && 'arn:aws:iam::827581582529:role/GitHubActionsRoleNextstrainBatchJobs' || '' }} | |
| # XXX TODO: <https://github.com/aws-actions/configure-aws-credentials#credential-lifetime> | |
| - name: Setup runtime ${{ inputs.runtime }} | |
| uses: ./.git/nextstrain/.github/actions/setup-nextstrain-cli | |
| with: | |
| cli-version: ">=7.1.0" | |
| runtime: ${{ inputs.runtime }} | |
| - id: attach | |
| name: Attach to AWS Batch job | |
| env: | |
| AWS_BATCH_JOB_ID: ${{ needs.run-build.outputs.AWS_BATCH_JOB_ID }} | |
| run: | | |
| # See <https://docs.github.com/en/actions/managing-workflow-runs/canceling-a-workflow#steps-github-takes-to-cancel-a-workflow-run> | |
| interrupt() { | |
| echo "Trapped SIGINT (job timed out/cancelled); detaching" >&2 | |
| # XXX FIXME: instead of converting SIGINT → SIGTSTP here, use --detach-on-interrupt in new release | |
| kill -SIGTSTP % | |
| wait % | |
| exit 0 | |
| } | |
| trap interrupt SIGINT | |
| nextstrain build \ | |
| --aws-batch \ | |
| --attach "$AWS_BATCH_JOB_ID" \ | |
| --no-download \ | |
| . \ | |
| & | |
| wait % | |
| # Allow the workflow to be considered successful even if this job errors | |
| # due to cancellation (timing out). Unfortunately, this doesn't | |
| # distinguish between error from cancellation and error from command | |
| # failure, so we work around that below. | |
| continue-on-error: true | |
| # Emit a "conclusion" output for the job that's based on the built-in | |
| # conclusion (success, failure, cancelled) of the "attach" step above. | |
| # This is the conclusion we care about for the job since the job's own | |
| # "conclusion" is masked/transformed by "continue-on-error: true" above. | |
| outputs: | |
| attach-step-conclusion: ${{ steps.attach.conclusion }} | |
| needs: [wait-3, run-build, workflow-context] | |
| if: needs.wait-3.outputs.attach-step-conclusion == 'cancelled' | |
| # Since the wait-N jobs use "continue-on-error: true" out of necessity (to | |
| # avoid failing the whole workflow when they time out and get cancelled), we | |
| # use a final job here to succeed or fail the whole workflow based on the | |
| # aggregate of their "attach" step conclusions. | |
| conclusion: | |
| needs: [wait-1, wait-2, wait-3, wait-4] | |
| if: always() | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: All attach steps in wait-N jobs were successful (or skipped) | |
| run: exit ${{ contains(needs.*.outputs.attach-step-conclusion, 'failure') && '1' || '0' }} | |
| # XXX TODO: Jobs can fall off the end of our wait-N chain and appear to be | |
| # successful/complete in GitHub but still running on AWS. Probably very | |
| # rare in reality, though, for an AWS job to take longer than 24h? | |
| # -trs, 12 Sept 2023 | |
| # Cancel the AWS Batch job if the GitHub workflow run is cancelled. | |
| cancellation: | |
| needs: [wait-4, run-build, workflow-context] | |
| if: cancelled() | |
| runs-on: ubuntu-latest | |
| steps: | |
| # Uses needs.workflow-context.outputs | |
| - # Need to run this after the build repo is cloned so that cloning the | |
| # build repo does not overwrite the .git dir and remove the extra support files | |
| # that we need from nextstrain/.github repo | |
| name: Checkout ${{ needs.workflow-context.outputs.repository }} (sha ${{ needs.workflow-context.outputs.sha }}) | |
| uses: actions/checkout@v3 | |
| with: | |
| repository: ${{ needs.workflow-context.outputs.repository }} | |
| ref: ${{ needs.workflow-context.outputs.sha }} | |
| path: ${{ env.NEXTSTRAIN_GITHUB_DIR }} | |
| - if: inputs.runtime == 'aws-batch' | |
| uses: aws-actions/configure-aws-credentials@v2 | |
| with: | |
| aws-region: us-east-1 | |
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | |
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | |
| role-to-assume: ${{ secrets.AWS_ACCESS_KEY_ID == '' && 'arn:aws:iam::827581582529:role/GitHubActionsRoleNextstrainBatchJobs' || '' }} | |
| # XXX TODO: <https://github.com/aws-actions/configure-aws-credentials#credential-lifetime> | |
| - name: Setup runtime ${{ inputs.runtime }} | |
| uses: ./.git/nextstrain/.github/actions/setup-nextstrain-cli | |
| with: | |
| cli-version: ">=7.1.0" | |
| runtime: ${{ inputs.runtime }} | |
| - id: cancel | |
| name: Cancel AWS Batch job | |
| env: | |
| AWS_BATCH_JOB_ID: ${{ needs.run-build.outputs.AWS_BATCH_JOB_ID }} | |
| run: | | |
| # XXX FIXME: instead of signaling here, use --cancel in new release | |
| set -x | |
| nextstrain build \ | |
| --aws-batch \ | |
| --attach "$AWS_BATCH_JOB_ID" \ | |
| --no-download \ | |
| . \ | |
| & | |
| # `nextstrain` will cancel the AWS Batch job upon receiving SIGINT | |
| # and stay attached while it waits for cancellation to occur, before | |
| # finally exiting non-zero. In the unlikely event that the job | |
| # completes before cancellation can occur, it'll exit 0, and we want | |
| # to treat that as an error. | |
| sleep 5 | |
| kill -SIGINT % | |
| sleep 1 | |
| kill -SIGINT % | |
| wait % && exit 1 || exit 0 | |
| # The cancellation job may fail, but we don't want that to impact the | |
| # overall workflow run status. | |
| continue-on-error: true |