pathogen-repo-build: Support assuming an AWS role via GitHub Actions' OIDC provider

[tsibley]

The default role has a pretty limited set of permissions right now: basically only those required to submit and interact with AWS Batch jobs, but not, for example, permissions to upload to our production S3 buckets. The permissions will probably need to be extended in the future as repos/builds opt into this workflow and this role assumption. The role's trust policy will also need to be extended to allow other repos to assume it.

While the role is currently manually managed, it should really be managed by a Terraform configuration (but one separate from the current nextstrain.org Terraform configurations).

Related-to: https://github.com/nextstrain/private/issues/22

Checklist

• Checks pass

[joverlee521]

The changes here look reasonable to me!

My main questions are about the permissions and management of the GitHubActionsRoleNextstrainBatchJobs role.

The default role has a pretty limited set of permissions right now: basically only those required to submit and interact with AWS Batch jobs, but not, for example, permissions to upload to our production S3 buckets.

Is the plan to manage upload permissions with another AWS Batch job role (as you mentioned previously). So the temp credentials here will allow the deployment and running of the Batch job, but permissions for AWS interactions within the build runtime will be managed by a separate role.

The role's trust policy will also need to be extended to allow other repos to assume it.

Is the plan to maintain a list of allowed repos in the role's trust policy and we would add repos to it as we add more pathogen workflows?

[tsibley]

moved to https://github.com/nextstrain/private/issues/96

Is the plan to manage upload permissions with another AWS Batch job role (as you mentioned previously). So the temp credentials here will allow the deployment and running of the Batch job, but permissions for AWS interactions within the build runtime will be managed by a separate role.

Unclear right now. A few options I see:

1. Extend the GitHubActionsRoleNextstrainBatchJobs role to allow uploading/downloading to S3 buckets used for pathogen builds (e.g. nextstrain-data, nextstrain-data-private, nextstrain-ncov-private, nextstrain-staging). Ensure the temporary credentials from role assumption are passed into the AWS Batch job via nextstrain build, as they are now.
2. Create a new role (or several) that allows the permissions above and have this workflow arrange to pass those temporary credentials into the Batch job (e.g. via nextstrain build + using output-credentials: true to configure-aws-credentials). This is a minor privilege separation and probably not worth it.
3. Update the default Batch job role (NextstrainJobsRole) to allow the permissions we need and stop passing credentials into the job via nextstrain build.
4. Create a new role (or several) that can be assumed by the Batch job itself and arrange via nextstrain build to have it assumed as part of the job submission.

Is the plan to maintain a list of allowed repos in the role's trust policy and we would add repos to it as we add more pathogen workflows?

I'd probably allow all existing pathogen repos initially, even if they're not using role assumption right now, and then yeah, add more as we need. I'd think of this as "enabling" a repo to use our AWS Batch infra.

[tsibley]

moved to https://github.com/nextstrain/private/issues/95

To choose amongst options, I'd think to consider:

• Review permissions we currently grant via individual IAM users/policies for each pathogen that's automated
• Decide if the union of those is palatable to grant widely to pathogen repos and any one with commit access to them
• Decide if the union of those is palatable to grant to any one who can submit AWS Batch jobs in our group (review this)
• Determine if the maximum session lifetime of 12 hours for an assumed role via GitHub OIDC is sufficient for all our pathogen builds or not

Option 1 is tempting because it's ~transparent to pathogen builds and easy. But it maybe runs into session lifetime issues if we're passing those assumed temporary credentials into the Batch job.

Options 3 and 4 are tempting because they're also ~transparent and easy and won't run into session lifetime issues. However, it means being ok allowing everyone in the lab who can submit jobs to make all the changes our automated builds need to do.

[tsibley]

This PR shouldn't have any functional change for our current usage, as I believe all callers provide explicit credentials themselves so the new role won't be assumed. As such, I'm inclined to merge it sooner than later to not leave it dangling. Further work to come will extend the role's permissions (or otherwise provide for additional permissions) and ensure they're passed in one way or another to the runtime.

[tsibley]

While the role is currently manually managed, it should really be managed by a Terraform configuration (but one separate from the current nextstrain.org Terraform configurations).

Will be part of nextstrain/infra@main...dev.