From 979905fe5291346a87af41b8df4486ca298e5c2f Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 13:16:35 -0700 Subject: [PATCH 01/14] wip --- .github/README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 .github/README.md diff --git a/.github/README.md b/.github/README.md new file mode 100644 index 00000000..8ebe2bd2 --- /dev/null +++ b/.github/README.md @@ -0,0 +1,21 @@ +# Current state of building and deploying ReportVision + +## Complete e2e ReportVision build and deploy + +If you would like to build and deploy ReporVsion all at once `deploy-dev.yml` will do the trick specifically for dev and demo environments within Azure only. There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account and update the AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_SUBSCRIPTION_ID secrets in Github Settings. + +When you deploy a `demo` environment specifically, note that the the `.github/actions/deploy-frontend/` Github Actions job will be skipped because for terraform will name the Azure Storage Account very uniquely. We decided to do this for demo environments because we are currently not using any type of DNS/custom domain names and wanted to make sure the user's could never return back to the specific url. Once the `dev-deploy.yml` run for a demo, just make sure to use the `build-deploy-frontend.yml` with the newly created Storage Account name from Azure. + +## Helm Install + +``` bash +helm install gitlab gitlab/gitlab -f values.yaml --namespace gitlab +``` + +## Sign into Gitlab + +To sign in, you must collect the password for the root user. This is automatically generated at installation time and stored in a Kubernetes Secret. Let’s fetch that password from the secret and decode it: + +```bash +kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo +``` \ No newline at end of file From 214d952c075fbcf2ffc6bba619ff9ae3f98a8c07 Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 14:58:43 -0700 Subject: [PATCH 02/14] wip --- .github/README.md | 62 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 49 insertions(+), 13 deletions(-) diff --git a/.github/README.md b/.github/README.md index 8ebe2bd2..16bc847b 100644 --- a/.github/README.md +++ b/.github/README.md @@ -1,21 +1,57 @@ -# Current state of building and deploying ReportVision +# Current Github Workflows for building and deploying ReportVision in Dev or Demo -## Complete e2e ReportVision build and deploy +## Prerequisites -If you would like to build and deploy ReporVsion all at once `deploy-dev.yml` will do the trick specifically for dev and demo environments within Azure only. There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account and update the AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_SUBSCRIPTION_ID secrets in Github Settings. +There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account and update the existing `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID` secrets in each Github Environmnet. -When you deploy a `demo` environment specifically, note that the the `.github/actions/deploy-frontend/` Github Actions job will be skipped because for terraform will name the Azure Storage Account very uniquely. We decided to do this for demo environments because we are currently not using any type of DNS/custom domain names and wanted to make sure the user's could never return back to the specific url. Once the `dev-deploy.yml` run for a demo, just make sure to use the `build-deploy-frontend.yml` with the newly created Storage Account name from Azure. +**NOTE**: Resource Groups were never created from Terraform on purpose to better replicate CDC's Azure setup and requirements for potential future migrations from Skylight's Azure. CDC manually created Resource Groups for us. -## Helm Install +Azure Resource Group Naming: -``` bash -helm install gitlab gitlab/gitlab -f values.yaml --namespace gitlab -``` +- `reportvision-rg-dev` +- `reportvision-rg-demo` -## Sign into Gitlab +## Complete e2e build and deploy for ReportVision -To sign in, you must collect the password for the root user. This is automatically generated at installation time and stored in a Kubernetes Secret. Let’s fetch that password from the secret and decode it: +If you would like to build and deploy all of ReportVisions services at once `deploy-dev.yml` will do the trick specifically for dev and demo environments within Azure only. -```bash -kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo -``` \ No newline at end of file +When you deploy a `demo` environment specifically, note that the `.github/actions/deploy-frontend/` Github Action will be skipped because Terraform will name the Azure Storage Account very uniquely. We decided to do this for demo environments because we are currently not using any type of DNS/custom domain names and wanted to make sure the user's could never return back to the specific url. Once the `deploy-dev.yml` completes for a demo, just make sure to use the `build-deploy-frontend.yml` with the newly created Storage Account name from Azure. + +Required Inputs: + +- `branch`: Any +- `deploy-env`: Drop down of available environments to chose from. +- `ocr-version`: This will be the Docker Tag. At the moment this caan be uniquely named and does not required any kind of format. We will also check if this tag already exists and if it does the docker build step will be skipped. + + +## Build and deploy the ReportVision's frontend only + +We made a seperate workflow that builds and deploys the frontend files only, `build-deploy-frontend.yml`. The reason for this is because currently the OCR-API docker build takes nearly an hour at times. Having to wait for that along with the Terraform setup job to complete, just to refresh the frontend can be a giant waste of time. Just make sure the Azure environment is already up from the `deploy-dev.yml` workflow or at the very least a Storage Account in is created. + +Required Inputs: + +- `deploy-env`: Drop down of available environments to chose from. + +Optional Inputs: + +- `storage-account-name`: When deploying a demo environment, you will need to use the unique Azure Storage Account name that was created and Terraform applied here. + +## Build and deploy the ReportVision's OCR-API only + +Just like with the frontend, we needed a way to refresh the OCR-API without having to re-apply Terraform and deploying the frontend. With `build-deploy-ocr.yml` we can build and publish a new OCR-API Docker image or we can use an already registered OCR-API Docker image. Once the Docker image is ready to go it will deploy it to an already running Azure App-Service Webapp. + +**Note**: Using an already registered Docker image will be MUCH faster than waiting for a new one to be built. + +Required Inputs: + +- `branch`: Any +- `deploy-env`: Drop down of available environments to chose from. +- `ocr-version`: This will be the Docker Tag. At the moment this caan be uniquely named and does not required any kind of format. We will also check if this tag already exists and if it does the docker build step will be skipped. + +# Github Workflows for building and deploying ReportVision in Staging or Production + +Unfortunately we never had the opportunity to pilot our amazing product to actual users which kept us from deploying to any type of Staging or Production environments. We also weren't entirely sure if we'd even be able to deploy to a centrally hosted Azure CP like the current one either. + +If we were able to deploy to a centrally hosted system. Our thought would have been building out a `deploy-stage.yml` workflow that is stuctured very similarily to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would get triggered. + +If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deploy as a container orchastrated system with something like Kubernetes, which would have paradigm shifted us completely. There was also the possibility of needing to do both. \ No newline at end of file From 7bd4020797a9ca4c0bafa5867e5fedcdfc3723b8 Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 15:02:52 -0700 Subject: [PATCH 03/14] wip --- .github/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/README.md b/.github/README.md index 16bc847b..fcff8ec2 100644 --- a/.github/README.md +++ b/.github/README.md @@ -2,7 +2,7 @@ ## Prerequisites -There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account and update the existing `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID` secrets in each Github Environmnet. +There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account, while also updating the existing `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID` secrets in each Github Environmnet. **NOTE**: Resource Groups were never created from Terraform on purpose to better replicate CDC's Azure setup and requirements for potential future migrations from Skylight's Azure. CDC manually created Resource Groups for us. From 7efc35b8f64205ff19b3bfb658f2fd7aca35c08e Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 15:04:07 -0700 Subject: [PATCH 04/14] wip --- .github/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/README.md b/.github/README.md index fcff8ec2..d5cc2eda 100644 --- a/.github/README.md +++ b/.github/README.md @@ -4,7 +4,7 @@ There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account, while also updating the existing `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID` secrets in each Github Environmnet. -**NOTE**: Resource Groups were never created from Terraform on purpose to better replicate CDC's Azure setup and requirements for potential future migrations from Skylight's Azure. CDC manually created Resource Groups for us. +**NOTE**: Resource Groups were never created from Terraform on purpose to better replicate CDC's Azure setup and requirements for potential future migrations from Skylight's Azure. CDC would manually create Resource Groups for us. Azure Resource Group Naming: From 5bb9a9f2835f7b7b8ad17150452064df7f8da372 Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 15:08:40 -0700 Subject: [PATCH 05/14] wip --- .github/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/README.md b/.github/README.md index d5cc2eda..c7470cd5 100644 --- a/.github/README.md +++ b/.github/README.md @@ -13,9 +13,9 @@ Azure Resource Group Naming: ## Complete e2e build and deploy for ReportVision -If you would like to build and deploy all of ReportVisions services at once `deploy-dev.yml` will do the trick specifically for dev and demo environments within Azure only. +If you would like to build and deploy all of ReportVision's services at once, `deploy-dev.yml` will do the trick specifically for dev and demo environments within Azure only. -When you deploy a `demo` environment specifically, note that the `.github/actions/deploy-frontend/` Github Action will be skipped because Terraform will name the Azure Storage Account very uniquely. We decided to do this for demo environments because we are currently not using any type of DNS/custom domain names and wanted to make sure the user's could never return back to the specific url. Once the `deploy-dev.yml` completes for a demo, just make sure to use the `build-deploy-frontend.yml` with the newly created Storage Account name from Azure. +When you deploy a `demo` environment specifically, note that the `.github/actions/deploy-frontend/` Github Action will be skipped because Terraform will name the Azure Storage Account very uniquely. We decided to do this for demo environments to ensure user's could never return back to the specific url. We are currently not using any custom domain names. Once the `deploy-dev.yml` completes for a demo, just make sure to use the `build-deploy-frontend.yml` with the newly created Storage Account name from Azure. Required Inputs: From f692337a56af32b4ae418d2b18c3fd24f48bf340 Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 15:20:05 -0700 Subject: [PATCH 06/14] wip --- .github/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/README.md b/.github/README.md index c7470cd5..72882267 100644 --- a/.github/README.md +++ b/.github/README.md @@ -15,7 +15,7 @@ Azure Resource Group Naming: If you would like to build and deploy all of ReportVision's services at once, `deploy-dev.yml` will do the trick specifically for dev and demo environments within Azure only. -When you deploy a `demo` environment specifically, note that the `.github/actions/deploy-frontend/` Github Action will be skipped because Terraform will name the Azure Storage Account very uniquely. We decided to do this for demo environments to ensure user's could never return back to the specific url. We are currently not using any custom domain names. Once the `deploy-dev.yml` completes for a demo, just make sure to use the `build-deploy-frontend.yml` with the newly created Storage Account name from Azure. +When you deploy a `demo` environment specifically, note that the `.github/actions/deploy-frontend/` Github Action will be skipped because Terraform will name the Azure Storage Account very uniquely. We decided to do this for demo environments to ensure user's could never return back to the specific url. We are currently not using any custom domain names. Once the `deploy-dev.yml` completes for a demo, just make sure to use the `build-deploy-frontend.yml` with the newly created Storage Account from Azure as an input. Required Inputs: @@ -34,11 +34,11 @@ Required Inputs: Optional Inputs: -- `storage-account-name`: When deploying a demo environment, you will need to use the unique Azure Storage Account name that was created and Terraform applied here. +- `storage-account-name`: After deploying a demo environment from `deploy-dev.yml`, you will need to use the unique Azure Storage Account name that was created here. ## Build and deploy the ReportVision's OCR-API only -Just like with the frontend, we needed a way to refresh the OCR-API without having to re-apply Terraform and deploying the frontend. With `build-deploy-ocr.yml` we can build and publish a new OCR-API Docker image or we can use an already registered OCR-API Docker image. Once the Docker image is ready to go it will deploy it to an already running Azure App-Service Webapp. +Just like with the frontend, we needed a way to refresh the OCR-API without having to re-apply Terraform and deploying the frontend. With `build-deploy-ocr.yml`, we can either build and publish a new OCR-API Docker image or we can use an already registered OCR-API Docker image. The OCR-API Docker images are located here: https://github.com/CDCgov/ReportVision/pkgs/container/reportvision-ocr-api. Once the Docker image is ready to go it will deploy it to the selected environments Azure App-Service Webapp. **Note**: Using an already registered Docker image will be MUCH faster than waiting for a new one to be built. @@ -50,8 +50,8 @@ Required Inputs: # Github Workflows for building and deploying ReportVision in Staging or Production -Unfortunately we never had the opportunity to pilot our amazing product to actual users which kept us from deploying to any type of Staging or Production environments. We also weren't entirely sure if we'd even be able to deploy to a centrally hosted Azure CP like the current one either. +Unfortunately we never had the opportunity to pilot our amazing product to actual users which kept us from deploying to any type of Staging or Production environments. We also weren't entirely sure if we'd even be able to deploy to a centrally hosted Azure account like our current one either. -If we were able to deploy to a centrally hosted system. Our thought would have been building out a `deploy-stage.yml` workflow that is stuctured very similarily to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would get triggered. +If we were able to deploy to a centrally hosted system. Our thought would have been to create a `deploy-stage.yml` workflow that is stuctured and functions very similarily to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would get triggered. -If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deploy as a container orchastrated system with something like Kubernetes, which would have paradigm shifted us completely. There was also the possibility of needing to do both. \ No newline at end of file +If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deployed as a container orchastrated system with tooling like Kubernetes. If this were to happen, we would have to paradigm shift completely. There was also the possibility of needing to do both. \ No newline at end of file From 7f9d5c417b3b971ac6bb44f36513ac58ced65532 Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 15:22:02 -0700 Subject: [PATCH 07/14] wip --- .github/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/README.md b/.github/README.md index 72882267..ef964718 100644 --- a/.github/README.md +++ b/.github/README.md @@ -30,6 +30,7 @@ We made a seperate workflow that builds and deploys the frontend files only, `bu Required Inputs: +- `branch`: Any - `deploy-env`: Drop down of available environments to chose from. Optional Inputs: From 7c1ac990b9632a524182e2b5c28a7ac4e93f452c Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 15:26:32 -0700 Subject: [PATCH 08/14] scope creep --- .github/workflows/build-deploy-frontend.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build-deploy-frontend.yml b/.github/workflows/build-deploy-frontend.yml index 5a1432e0..166a8fa6 100644 --- a/.github/workflows/build-deploy-frontend.yml +++ b/.github/workflows/build-deploy-frontend.yml @@ -32,7 +32,7 @@ jobs: frontend-build-path: ./frontend/dist/ node-version: 20 - deploy-with-blob-name-optional: + deploy-frontend: name: Deploy runs-on: ubuntu-latest environment: ${{ inputs.deploy-env }} From 102219dd674cab4cefb18ed40eac6fee9fbe8631 Mon Sep 17 00:00:00 2001 From: Administrator Date: Tue, 5 Nov 2024 15:27:53 -0700 Subject: [PATCH 09/14] scope creep --- .github/workflows/build-deploy-frontend.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build-deploy-frontend.yml b/.github/workflows/build-deploy-frontend.yml index 166a8fa6..a22d59c4 100644 --- a/.github/workflows/build-deploy-frontend.yml +++ b/.github/workflows/build-deploy-frontend.yml @@ -11,7 +11,7 @@ on: - dev - demo storage-account-name: - description: 'After the demo env gets created, copy its blob storage name here' + description: 'After the demo env gets created, copy its Azure Storage Account name here' required: false permissions: From 867c6bd96f736cd5a6e36eaf9492cd99bc99ad32 Mon Sep 17 00:00:00 2001 From: Administrator Date: Mon, 18 Nov 2024 13:14:45 -0700 Subject: [PATCH 10/14] wip --- ops/terraform/README.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 ops/terraform/README.md diff --git a/ops/terraform/README.md b/ops/terraform/README.md new file mode 100644 index 00000000..3743ec66 --- /dev/null +++ b/ops/terraform/README.md @@ -0,0 +1,14 @@ +# ReportVision's Terraform Setup + +Currently, our infrastructure is built specifically for Azure, with a traditional cloud architecture hosting our frontend code from blob storage and our OCR-API running in an App Service. Both the frontend and the OCR-API are behind a Virtual Network and load balanced by an App Gateway. + + +## Prerequisites + +When using Terraform, you will need to created a `terraform.tfvars` file in the `ops/terraform` directory with variables: + +``` bash + resource_group_name = "reportvision-rg-" + name = "reportvision" + sku_name = "S2" +``` \ No newline at end of file From 0be0f01354a1015f16ded3226333828ec1820610 Mon Sep 17 00:00:00 2001 From: Administrator Date: Mon, 18 Nov 2024 13:15:22 -0700 Subject: [PATCH 11/14] wip --- .github/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/README.md b/.github/README.md index ef964718..b9ce89f6 100644 --- a/.github/README.md +++ b/.github/README.md @@ -2,7 +2,7 @@ ## Prerequisites -There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account, while also updating the existing `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID` secrets in each Github Environmnet. +There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account, while also updating the existing `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID` secrets in each Github Environment. **NOTE**: Resource Groups were never created from Terraform on purpose to better replicate CDC's Azure setup and requirements for potential future migrations from Skylight's Azure. CDC would manually create Resource Groups for us. @@ -26,7 +26,7 @@ Required Inputs: ## Build and deploy the ReportVision's frontend only -We made a seperate workflow that builds and deploys the frontend files only, `build-deploy-frontend.yml`. The reason for this is because currently the OCR-API docker build takes nearly an hour at times. Having to wait for that along with the Terraform setup job to complete, just to refresh the frontend can be a giant waste of time. Just make sure the Azure environment is already up from the `deploy-dev.yml` workflow or at the very least a Storage Account in is created. +We made a separate workflow that builds and deploys the frontend files only, `build-deploy-frontend.yml`. The reason for this is because currently the OCR-API docker build takes nearly an hour at times. Having to wait for that along with the Terraform setup job to complete, just to refresh the frontend can be a giant waste of time. Just make sure the Azure environment is already up from the `deploy-dev.yml` workflow or at the very least a Storage Account in is created. Required Inputs: @@ -53,6 +53,6 @@ Required Inputs: Unfortunately we never had the opportunity to pilot our amazing product to actual users which kept us from deploying to any type of Staging or Production environments. We also weren't entirely sure if we'd even be able to deploy to a centrally hosted Azure account like our current one either. -If we were able to deploy to a centrally hosted system. Our thought would have been to create a `deploy-stage.yml` workflow that is stuctured and functions very similarily to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would get triggered. +If we were able to deploy to a centrally hosted system. Our thought would have been to create a `deploy-stage.yml` workflow that is structured and functions very similarly to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would get triggered. -If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deployed as a container orchastrated system with tooling like Kubernetes. If this were to happen, we would have to paradigm shift completely. There was also the possibility of needing to do both. \ No newline at end of file +If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deployed as a container orchestrated system with tooling like Kubernetes. If this were to happen, we would have to paradigm shift completely. There was also the possibility of needing to do both. \ No newline at end of file From 259c43be280345b87d56d7f35f5ed22afbeef6a4 Mon Sep 17 00:00:00 2001 From: Administrator Date: Mon, 6 Jan 2025 10:49:02 -0700 Subject: [PATCH 12/14] updates --- .github/README.md | 33 ++++++++++++++++++-------------- .github/workflows/deploy-dev.yml | 4 ++-- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/.github/README.md b/.github/README.md index b9ce89f6..f6622d1d 100644 --- a/.github/README.md +++ b/.github/README.md @@ -2,7 +2,7 @@ ## Prerequisites -There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account, while also updating the existing `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID` secrets in each Github Environment. +There are secrets for Azure authentication from Github Action's located within the Github Settings. At the time of reading this, you may need to create new federated secrets and Resource Groups in your Azure account, while also updating the existing `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID`, `AZURE_OBJECT_ID` secrets in each Github Environment. **NOTE**: Resource Groups were never created from Terraform on purpose to better replicate CDC's Azure setup and requirements for potential future migrations from Skylight's Azure. CDC would manually create Resource Groups for us. @@ -13,46 +13,51 @@ Azure Resource Group Naming: ## Complete e2e build and deploy for ReportVision -If you would like to build and deploy all of ReportVision's services at once, `deploy-dev.yml` will do the trick specifically for dev and demo environments within Azure only. - -When you deploy a `demo` environment specifically, note that the `.github/actions/deploy-frontend/` Github Action will be skipped because Terraform will name the Azure Storage Account very uniquely. We decided to do this for demo environments to ensure user's could never return back to the specific url. We are currently not using any custom domain names. Once the `deploy-dev.yml` completes for a demo, just make sure to use the `build-deploy-frontend.yml` with the newly created Storage Account from Azure as an input. +To build and deploy all of ReportVision's services at once, `deploy-dev.yml` will do the trick specifically for dev environments within Azure only. Required Inputs: - `branch`: Any - `deploy-env`: Drop down of available environments to chose from. -- `ocr-version`: This will be the Docker Tag. At the moment this caan be uniquely named and does not required any kind of format. We will also check if this tag already exists and if it does the docker build step will be skipped. + +Optional Inputs: + +- `ocr-docker-tag`: The OCR Docker Tag. This input is optional because if it is left blank, the workflow will build and publish a new docker image each time. If you would like to deploy a previously built docker image, you can add the tag here. +- `middleware-docker-tag`: The Middleware Docker Tag. This input is optional because if it is left blank, the workflow will build and publish a new docker image each time. If you would like to deploy a previously built docker image, you can add the tag here. ## Build and deploy the ReportVision's frontend only -We made a separate workflow that builds and deploys the frontend files only, `build-deploy-frontend.yml`. The reason for this is because currently the OCR-API docker build takes nearly an hour at times. Having to wait for that along with the Terraform setup job to complete, just to refresh the frontend can be a giant waste of time. Just make sure the Azure environment is already up from the `deploy-dev.yml` workflow or at the very least a Storage Account in is created. +We made a separate workflow that builds and deploys the frontend files only, `build-deploy-frontend.yml`. Having to wait for the end-to-end deploy, along with the Terraform setup job to complete, just to refresh the frontend can be a giant waste of time. Just make sure the Azure environment is already up from the `deploy-dev.yml` workflow or at the very least a Storage Account in is created. Required Inputs: - `branch`: Any - `deploy-env`: Drop down of available environments to chose from. -Optional Inputs: +## Build and deploy the ReportVision's OCR-API and Middleware-API separately -- `storage-account-name`: After deploying a demo environment from `deploy-dev.yml`, you will need to use the unique Azure Storage Account name that was created here. +Just like with the frontend, we needed a way to refresh the OCR-API and/or Middleware-API without having to re-apply Terraform and deploying the frontend. With `build-deploy-ocr.yml` or `build-deploy-ocr.yml` we can either build and publish a new docker image or we can use an already registered docker image. Both the OCR-API and Middleware-API docker images are published here: +- https://github.com/CDCgov/ReportVision/pkgs/container/reportvision-ocr +- https://github.com/CDCgov/ReportVision/pkgs/container/reportvision-middleware. -## Build and deploy the ReportVision's OCR-API only +Once the workflow builds and publishes the images, it will deploy it to the selected environments Azure App-Service Webapp. -Just like with the frontend, we needed a way to refresh the OCR-API without having to re-apply Terraform and deploying the frontend. With `build-deploy-ocr.yml`, we can either build and publish a new OCR-API Docker image or we can use an already registered OCR-API Docker image. The OCR-API Docker images are located here: https://github.com/CDCgov/ReportVision/pkgs/container/reportvision-ocr-api. Once the Docker image is ready to go it will deploy it to the selected environments Azure App-Service Webapp. - -**Note**: Using an already registered Docker image will be MUCH faster than waiting for a new one to be built. +**Note**: Using an already registered docker image will be a bit faster than waiting for a new one to be built. Also, this is how we quickly rollback versions. Required Inputs: - `branch`: Any - `deploy-env`: Drop down of available environments to chose from. -- `ocr-version`: This will be the Docker Tag. At the moment this caan be uniquely named and does not required any kind of format. We will also check if this tag already exists and if it does the docker build step will be skipped. + +Optional Inputs: + +- `ocr-docker-tag`: The OCR Docker Tag. This input is optional because if it is left blank, the workflow will build and publish a new docker image each time. If you would like to deploy a previously built docker image, you can add the tag here. # Github Workflows for building and deploying ReportVision in Staging or Production Unfortunately we never had the opportunity to pilot our amazing product to actual users which kept us from deploying to any type of Staging or Production environments. We also weren't entirely sure if we'd even be able to deploy to a centrally hosted Azure account like our current one either. -If we were able to deploy to a centrally hosted system. Our thought would have been to create a `deploy-stage.yml` workflow that is structured and functions very similarly to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would get triggered. +If we were able to deploy to a centrally hosted system. Our thought would have been to create a `deploy-stage.yml` workflow that is structured and functions very similarly to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would then get triggered. If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deployed as a container orchestrated system with tooling like Kubernetes. If this were to happen, we would have to paradigm shift completely. There was also the possibility of needing to do both. \ No newline at end of file diff --git a/.github/workflows/deploy-dev.yml b/.github/workflows/deploy-dev.yml index f1dba2d5..d93e4bed 100644 --- a/.github/workflows/deploy-dev.yml +++ b/.github/workflows/deploy-dev.yml @@ -10,10 +10,10 @@ on: options: - dev ocr-docker-tag: - description: 'This is optional if you would like to deploy an already published OCR-API image' + description: 'OCR Docker Tag (This is optional. If you would like to deploy an already published OCR-API image):' required: false middleware-docker-tag: - description: 'This is optional if you would like to deploy an already published Middleware-API image' + description: 'Middleware Docker Tag (This is optional. If you would like to deploy an already published Middleware-API image):' required: false permissions: From 3f3c7e3058455f7a91fdfd661c4b55b096b6fabf Mon Sep 17 00:00:00 2001 From: Administrator Date: Mon, 6 Jan 2025 10:56:21 -0700 Subject: [PATCH 13/14] updates --- .github/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/README.md b/.github/README.md index f6622d1d..586479db 100644 --- a/.github/README.md +++ b/.github/README.md @@ -60,4 +60,4 @@ Unfortunately we never had the opportunity to pilot our amazing product to actua If we were able to deploy to a centrally hosted system. Our thought would have been to create a `deploy-stage.yml` workflow that is structured and functions very similarly to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would then get triggered. -If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deployed as a container orchestrated system with tooling like Kubernetes. If this were to happen, we would have to paradigm shift completely. There was also the possibility of needing to do both. \ No newline at end of file +If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deployed as a container orchestrated system with tooling like Kubernetes. This would make it easier for us to be cloud-agnostic and have the ability to quickly "lift-and-shift" our product into different organizations. If this were to happen, we would had to paradigm shift completely. There was also the possibility of needing to do both for demo, internal and centrally hosted options for STLT customers. \ No newline at end of file From 5d83e777e046348b7ec7caa165605ddd0d8b6e71 Mon Sep 17 00:00:00 2001 From: Administrator Date: Mon, 6 Jan 2025 10:59:46 -0700 Subject: [PATCH 14/14] updates --- .github/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/README.md b/.github/README.md index 586479db..f4a4ea3e 100644 --- a/.github/README.md +++ b/.github/README.md @@ -60,4 +60,4 @@ Unfortunately we never had the opportunity to pilot our amazing product to actua If we were able to deploy to a centrally hosted system. Our thought would have been to create a `deploy-stage.yml` workflow that is structured and functions very similarly to `deploy-dev.yml`, except it would be triggered off of the `main` branch or Github `tags`. If all staging jobs and tests pass, a `deploy-prod.yml` workflow would then get triggered. -If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deployed as a container orchestrated system with tooling like Kubernetes. This would make it easier for us to be cloud-agnostic and have the ability to quickly "lift-and-shift" our product into different organizations. If this were to happen, we would had to paradigm shift completely. There was also the possibility of needing to do both for demo, internal and centrally hosted options for STLT customers. \ No newline at end of file +If we were required to deploy to STLT-hosted environments, our plan was going to ensure that all services are containerized and deployed as a container orchestrated system with tooling like Kubernetes. This would make it easier for us to be cloud-agnostic and have the ability to quickly "lift-and-shift" our product into different organizations. If this were to happen, we would had to paradigm shift completely. \ No newline at end of file