Prefixing environments for mono-repo setup

[bzurkowski]

Details

I handle infrastructure code (Terraform) in a mono-repo, which necessitates the deployment of numerous projects across various environments from a single repository.

I managed to implement a separate branch-deployment gate for each project using distinct triggers:

trigger:
  if: |
    github.event.issue.pull_request
  runs-on: ubuntu-latest
  steps:
    - name: Gate branch deployment
      id: branch-deploy
      uses: github/[email protected]
      with:
        trigger: .deploy ${{ inputs.project_name }}
        noop_trigger: plan

The above configuration produces the following IssueOps interface:

.deploy foo dev/staging/prod
.deploy foo plan dev/staging/prod

.deploy bar dev/staging/prod
.deploy bar plan dev/staging/prod

However, given multiple projects, specified environments, i.e., dev/staging/prod, are "shared" (from the branch-deploy action's perspective), which implies that whenever I deploy to project foo, corresponding Github environments are being locked and altered with deployment status, preventing me against deploying to the boo project from other pull requests.

Consequently, in a mono-repo setup, one must distinguish Github environments by project, e.g., using a prefix: foo-dev vs. bar-dev, foo-prod vs. bar-prod, etc.

That is possible to implement by a combination of environment and environment_targets input variables:

environment: foo-dev
environment_targets: foo-dev,foo-staging,foo-prod

However, with the current design of the branch-deploy action, the IssueOps interface would look like this:

.deploy foo foo-dev
.deploy bar bar-dev

... which seems inconvenient and unreadable as one has to repeat the project name twice.

I wonder if we could introduce an additional parameter, e.g., environment_prefix, that would alter the behavior of the branch-deploy action internally so that the action, given an environment dev (from issue comment), prefixes it internally with environment_prefix and only the prefixed Github environment is used for further processing. In other words, it would be great to somehow separate input environments from Github environments.

environment: dev
environment_targets: dev,staging,prod
environment_prefix: foo

That would be a significant enabler towards adopting the action into existing mono-repo configurations.

[GrantBirki]

@bzurkowski I'm not sure if this would exactly solve the issue at hand, but have you looked at using parameters by any chance? I recently added this feature to the Action and I'm wondering if you could pass in details specific to your environment in your monorepo to gain better control over your deployments in that regards.

Take a look and let me know what ya think! It may work for you or it might not. Worth a read though I think.

[bzurkowski]

@GrantBirki I am aware of the parameters feature, but I do not find it sufficient for my case as it does not provide a more readable interface than the one I struggle with right now, i.e., .deploy app | ENV=dev vs. .deploy app app-dev. Additionally, I perceive the environment as a primary argument and the command subject, whereas parameters seem to serve more as options that modify the deployment to a certain degree (akin to the distinction between positional arguments and options).

I am trying hard to integrate the branch-deploy action into the mono-repo setup, and I think I am close to achieving a nice process for infrastructure deployment that could be competitive against alternatives.

What I need is a mechanism that allows supplying a simple, unprefixed list of environments to the action, such as environment_targets: "dev,staging,prod". However, the action should handle these as project-namespaced environments, meaning they should be internally processed as app-dev,app-staging,app-prod.

That would make a convenient interface for multi-project configurations:

- name: Gate branch deployment
  id: branch-deploy
  uses: github/[email protected]
  with:
    trigger: .deploy ${{ inputs.project_name }}
    lock_trigger: .lock ${{ inputs.project_name }}
    unlock_trigger: .unlock ${{ inputs.project_name }}
    help_trigger: .help ${{ inputs.project_name }}
    noop_trigger: .plan ${{ inputs.project_name }}
    environment_prefix: ${{ inputs.project_name }}
    environment_targets: ${{ inputs.environments }}

Then, one could run gates for multiple projects in a matrix like this:

  deploy:
    strategy:
      matrix:
        include:
        - project_name: app
          environments: dev,staging,prod
        - project_name: cicd
          environments: dev,prod
        ...

Alternatively, we could consider adding a namespace or project input variable to make the interface even more adapted to multi-project configurations.

I am uncertain whether my suggestion aligns with your envisioned development path for the action. However, after reviewing issues in this repository, it appears some users are exploring similar adjustments due to their mono-repo setups (e.g., #91).

[bzurkowski]

@GrantBirki Do you have any opinion on the above idea? Does my suggestion align with your envisioned development path for the action? Do you need more context (or sample configuration) to better evaluate the idea?

[GrantBirki]

If you have any more context or sample configuration, I will take all the details I can get! I'm trying to wrap all this together in my head and figure out a path forward to make monorepos better supported when using this Action.

[GrantBirki]

@bzurkowski 👋 hey again! Did you ever end up collecting additional information or did you perhaps find a work-around for your situation? Curious as to what you finally ended up doing for a solution to this, thanks! 🙇

[bzurkowski]

@GrantBirki Currently, we successfully employ the mono-repo approach with this action (10 infrastructure projects within a single repo) accepting the drawbacks of the .deploy <project> <project>-<env> IssueOps interface. However, this creates some confusion and mistakes and we definitely need some improvement. For now, I do not see an option other than enhancing the action interface as proposed in #168 (comment) and #168 (comment). I will paste our configuration soon. Recently, I was very low on capacity hence no response for a long time, sorry 😢

[redoz]

It would maybe be even more flexible if instead of a prefix, a mapping could be provided?

[Ugzuzg]

My ideal approach for a monorepo would be to still use .deploy development or .lock development without specifying a project, and let the action figure out which projects are affected by this change and deploy or lock the environments specific to these projects (app1-development, app2-development).

That way, no one has to keep track of which projects have changed in a specific PR to deploy them (and avoid an accidental issue of forgetting to deploy something before merging), and allow two concurrent deploys from different PRs, if they are affecting different projects.

[GrantBirki]

I am very open to feedback or suggestions on how to implement this best. I personally haven't ever used this Action in a monorepo setup so I don't actually have all that much insight into how this may work best for users.