Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pathogen-repo-build: Move build.log to ignored directory #78

Merged
merged 2 commits into from
Apr 4, 2024

Conversation

joverlee521
Copy link
Contributor

@joverlee521 joverlee521 commented Mar 15, 2024

Description of proposed changes

See commit for details.

Related issue(s)

Resolves #77

Checklist

  • Checks pass

The entire working directory gets zipped and uploaded to AWS Batch when
using the `aws-batch` runtime, which includes an early `build.log` file.
When the build completes and downloads the build outputs, the early
`build.log` overwrites the local `build.log` of the GH Action.
This results in a truncated file that does not include the
AWS_BATCH_JOB_ID and prevents the generation of the AWS Batch summary
and the re-attachment to the AWS Batch job in subsequent `wait` jobs.

This commit moves the hard-coded `build.log` file to an ignored directory
(the same directory that we are using for the nextstrain/.github repo¹).
This prevents the `build.log` from being uploaded with the build.

Previously tried other paths, but they all caused errors.
- "../build.log" - ".." pattern is not allowed for the `upload-artifact` action²
- "~/build.log" - `hashfiles` requires files to be in the `GITHUB_WORKSPACE`³

¹ #44 (comment)
² actions/upload-artifact#176
³ https://docs.github.com/en/actions/learn-github-actions/expressions#hashfiles
@joverlee521
Copy link
Contributor Author

The CI workflow included the AWS Batch job summary 🎉

The CI workflow now reveals a separate issue...In the CI workflow, the aws-batch build is run without the --detach flag and is able to complete the build within the run-build job. However, the wait-1 job still re-attaches to the AWS Batch job because it's only run condition is the existence of the AWS_BATCH_JOB_ID. This is not a big deal since the wait-1 job will only take a couple seconds to download a completed build and our automated workflows usually use the --detach flag.

I don't think it's worth complicating the workflow logic just to prevent an extra wait-1 job, but please chime in if there's a good reason to update it!

@joverlee521 joverlee521 requested a review from tsibley March 15, 2024 21:23
Taking @victorlin's suggestion¹ to keep the `.git/build.log` as an
implementation detail within the workflow.

¹ #78 (comment)
@joverlee521
Copy link
Contributor Author

Merging since I intend to do some additional work on the pathogen-repo-build workflow to support role assumption in other runtimes.

@joverlee521 joverlee521 merged commit d15f4f9 into master Apr 4, 2024
39 of 41 checks passed
@joverlee521 joverlee521 deleted the fix-ci-aws-batch branch April 4, 2024 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: pathogen-repo-build (aws-batch) does not output full logs to build.log
2 participants