Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add middleware to App Gateway #458

Merged
merged 29 commits into from
Dec 10, 2024
Merged

Add middleware to App Gateway #458

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
28ef475
Add middleware to app gateway
Dec 5, 2024
6ebfb09
fmt
Dec 5, 2024
f162efa
push middleware image to dev
Dec 5, 2024
3a879f8
wip
Dec 6, 2024
1aee8f9
wip
Dec 6, 2024
97a82e0
wip
Dec 6, 2024
bddcb8d
wip
Dec 6, 2024
6e4c811
wrong image
Dec 6, 2024
93f4f86
wrong image
Dec 6, 2024
eb399bf
rm entrypoint
Dec 6, 2024
d81fa29
try diff dockerfile fmt
Dec 6, 2024
f6535bb
try diff dockerfile fmt
Dec 6, 2024
cb88e00
try diff dockerfile fmt
Dec 6, 2024
bd9202c
try diff dockerfile fmt
Dec 6, 2024
30a3b49
try diff dockerfile fmt
Dec 6, 2024
d894ee8
try diff dockerfile fmt
Dec 6, 2024
9cbab03
try diff dockerfile fmt
Dec 6, 2024
f55ed1f
try diff dockerfile fmt
Dec 6, 2024
e3fb8f5
try diff dockerfile fmt
Dec 6, 2024
6dcd27b
try diff dockerfile fmt
Dec 6, 2024
636719a
try diff dockerfile fmt
Dec 6, 2024
4e422b8
try diff dockerfile fmt
Dec 6, 2024
02247a3
try diff dockerfile fmt
Dec 6, 2024
5cc093e
try diff dockerfile fmt
Dec 6, 2024
52f1179
try diff dockerfile fmt
Dec 6, 2024
7d3a15b
clean
Dec 9, 2024
b29e740
clean
Dec 9, 2024
8dd3531
clean
Dec 9, 2024
aa6f7e1
Merge branch 'main' into put-middle-behind-app-gateway
Dec 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 21 additions & 19 deletions .github/workflows/build-deploy-ocr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
name: Create, publish, deploy a OCR API image

on:
workflow_dispatch:
inputs:
deploy-env:
description: 'The environment to deploy to'
required: true
type: choice
options:
- dev
- demo
ocr-docker-tag:
description: 'This is optional if you would like to deploy an already published OCR-API image'
required: false
push:
branches: put-middle-behind-app-gateway
# workflow_dispatch:
# inputs:
# deploy-env:
# description: 'The environment to deploy to'
# required: true
# type: choice
# options:
# - dev
# - demo
# ocr-docker-tag:
# description: 'This is optional if you would like to deploy an already published OCR-API image'
# required: false

permissions:
contents: read
Expand All @@ -35,7 +37,7 @@ jobs:
shell: bash
id: set_tag
run: |
USER_INPUT="${{ github.event.inputs.ocr-docker-tag }}"
USER_INPUT=""
echo "docker_tag=$(
#this gives a new sha as default if dispatch input is empty
echo ${USER_INPUT:-"${{ github.sha }}"}
Expand All @@ -44,7 +46,7 @@ jobs:
id: image_check
run: |
echo "docker_inspect=$(
docker manifest inspect ghcr.io/${{ env.REPO }}-ocr:${{ steps.set_tag.outputs.docker_tag }} > /dev/null ; echo $?
docker manifest inspect ghcr.io/${{ env.REPO }}-middleware:${{ steps.set_tag.outputs.docker_tag }} > /dev/null ; echo $?
)" >> $GITHUB_OUTPUT
- name: Build and Push backend
if: ${{ steps.image_check.outputs.docker_inspect == 1 }}
Expand All @@ -54,14 +56,14 @@ jobs:
docker-pw: ${{ secrets.GITHUB_TOKEN }}
docker-username: ${{ github.actor }}
docker-tag: ${{ steps.set_tag.outputs.docker_tag }}
dockerfile-path: ./OCR/Dockerfile
docker-context-path: ./OCR/
dockerfile-path: ./backend/Dockerfile
docker-context-path: ./backend/
api-name: ocr

deploy-ocr:
name: Deploy OCR
runs-on: ubuntu-latest
environment: ${{ inputs.deploy-env }}
environment: dev
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to hardcode dev here? Or can we make dynamic so if whomever picks it up can select dev, as well as, stage or prod (after they build out those environments)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is how we do local github action testing. I will make this a draft

Copy link
Collaborator Author

@derekadombek derekadombek Dec 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also wouldnt need to test like this if we create a seperate worflow for building and deploying the middleware

needs: [build-publish-ocr]
steps:
- uses: actions/checkout@v4
Expand All @@ -73,7 +75,7 @@ jobs:
- name: Deploy OCR-API
uses: ./.github/actions/deploy-api
with:
deploy-env: ${{ inputs.deploy-env }}
deploy-env: dev
docker-tag: ${{ needs.build-publish-ocr.outputs.docker_tag }}
docker-registry: ghcr.io
api-name: ocr
api-name: middleware
2 changes: 2 additions & 0 deletions backend/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ FROM amazoncorretto:17
ENV HOME=/app
RUN mkdir -p $HOME
WORKDIR $HOME

ENTRYPOINT ["./gradlew", "bootRun", "--continuous", "--args='--server.port=8081'"]
19 changes: 15 additions & 4 deletions ops/terraform/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,9 @@ module "app_gateway" {
tags = local.management_tags
env = local.environment

fqdns = module.ocr_api.app_hostname
depends_on = [module.networking, module.ocr_api]
fqdns_ocr = module.ocr_api.app_hostname
fqdns_middleware = module.middleware_api.app_hostname
depends_on = [module.networking, module.ocr_api, module.middleware_api]
}

module "storage" {
Expand All @@ -67,12 +68,17 @@ module "middleware_api" {
resource_group = data.azurerm_resource_group.rg.name
app_subnet_id = module.networking.middlewaresubnet_id

app_settings = {
WEBSITES_PORT = "8081"
}

lb_subnet_id = module.networking.lbsubnet_id
health_path = "/actuator/health"
env = local.environment
vnet = module.networking.network_name
sku_name = var.sku_name
https_only = true
depends_on = [module.networking.middlewaresubnet_id, module.networking.lbsubnet_id]
depends_on = [module.networking.middlewaresubnet_id, module.networking.lbsubnet_id]
}

module "ocr_api" {
Expand All @@ -82,12 +88,17 @@ module "ocr_api" {
location = data.azurerm_resource_group.rg.location
resource_group = data.azurerm_resource_group.rg.name
app_subnet_id = module.networking.ocrsubnet_id

app_settings = {
WEBSITES_PORT = "8000"
}

lb_subnet_id = module.networking.middlewaresubnet_id
env = local.environment
vnet = module.networking.network_name
sku_name = var.sku_name
https_only = true
depends_on = [module.networking.ocrsubnet_id, module.networking.middlewaresubnet_id]
depends_on = [module.networking.ocrsubnet_id, module.networking.middlewaresubnet_id]
}

module "ocr_autoscale" {
Expand Down
154 changes: 121 additions & 33 deletions ops/terraform/modules/app_gateway/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,27 @@ resource "azurerm_public_ip" "lb-pip" {

# since these variables are re-used - a locals block makes this more maintainable
locals {
backend_address_pool_name_static = "${var.name}-${var.env}-beap-static"
backend_address_pool_name_api = "${var.name}-${var.env}-beap-api"
frontend_port_name_api = "${var.name}-${var.env}-feport-api"
frontend_port_name_static = "${var.name}-${var.env}-feport-static"
frontend_ip_configuration_name = "${var.name}-${var.env}-feip"
http_setting_name_static = "${var.name}-${var.env}-be-htst-static"
http_setting_name_api = "${var.name}-${var.env}-be-htst-api"
listener_name_static = "${var.name}-${var.env}-httplstn-static"
listener_name_api = "${var.name}-${var.env}-httplstn-api"
request_routing_rule_name_api = "${var.name}-${var.env}-rqrt-api"
request_routing_rule_name_static = "${var.name}-${var.env}-rqrt-static"
redirect_configuration_name = "${var.name}-${var.env}-rdrcfg"
static_probe_name_app = "${var.name}-${var.env}-be-probe-app-static"
api_probe_name_app = "${var.name}-${var.env}-be-probe-app-api"
redirect_rule = "${var.name}-${var.env}-redirect"
backend_address_pool_name_static = "${var.name}-${var.env}-beap-static"
backend_address_pool_name_api_ocr = "${var.name}-${var.env}-beap-api-ocr"
backend_address_pool_name_api_middleware = "${var.name}-${var.env}-beap-api-middleware"
frontend_port_name_api_ocr = "${var.name}-${var.env}-feport-api-ocr"
frontend_port_name_api_middleware = "${var.name}-${var.env}-feport-api-middleware"
frontend_port_name_static = "${var.name}-${var.env}-feport-static"
frontend_ip_configuration_name = "${var.name}-${var.env}-feip"
http_setting_name_static = "${var.name}-${var.env}-be-htst-static"
http_setting_name_api_ocr = "${var.name}-${var.env}-be-htst-api-ocr"
http_setting_name_api_middleware = "${var.name}-${var.env}-be-htst-api-middleware"
listener_name_static = "${var.name}-${var.env}-httplstn-static"
listener_name_api_ocr = "${var.name}-${var.env}-httplstn-api-ocr"
listener_name_api_middleware = "${var.name}-${var.env}-httplstn-api-middleware"
request_routing_rule_name_api_ocr = "${var.name}-${var.env}-rqrt-api-ocr"
request_routing_rule_name_api_middleware = "${var.name}-${var.env}-rqrt-api-middleware"
request_routing_rule_name_static = "${var.name}-${var.env}-rqrt-static"
redirect_configuration_name = "${var.name}-${var.env}-rdrcfg"
static_probe_name_app = "${var.name}-${var.env}-be-probe-app-static"
api_probe_name_app_ocr = "${var.name}-${var.env}-be-probe-app-api-ocr"
api_probe_name_app_middleware = "${var.name}-${var.env}-be-probe-app-api-middleware"
redirect_rule = "${var.name}-${var.env}-redirect"
}

resource "azurerm_application_gateway" "load_balancer" {
Expand All @@ -35,7 +41,6 @@ resource "azurerm_application_gateway" "load_balancer" {
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}

gateway_ip_configuration {
Expand Down Expand Up @@ -72,23 +77,23 @@ resource "azurerm_application_gateway" "load_balancer" {

# ------- OCR API -------------------------
backend_address_pool {
name = local.backend_address_pool_name_api
fqdns = [var.fqdns]
name = local.backend_address_pool_name_api_ocr
fqdns = [var.fqdns_ocr]
ip_addresses = var.ip_addresses
}

backend_http_settings {
name = local.http_setting_name_api
name = local.http_setting_name_api_ocr
cookie_based_affinity = "Disabled"
port = 443
protocol = "Https"
request_timeout = 120
pick_host_name_from_backend_address = true
probe_name = local.api_probe_name_app
probe_name = local.api_probe_name_app_ocr
}

probe {
name = local.api_probe_name_app
name = local.api_probe_name_app_ocr
interval = 30
timeout = 30
unhealthy_threshold = 3
Expand All @@ -102,6 +107,38 @@ resource "azurerm_application_gateway" "load_balancer" {
}
}

# ------- Middleware API -------------------------
backend_address_pool {
name = local.backend_address_pool_name_api_middleware
fqdns = [var.fqdns_middleware]
ip_addresses = var.ip_addresses
}

backend_http_settings {
name = local.http_setting_name_api_middleware
cookie_based_affinity = "Disabled"
port = 443
protocol = "Https"
request_timeout = 120
pick_host_name_from_backend_address = true
probe_name = local.api_probe_name_app_middleware
}

probe {
name = local.api_probe_name_app_middleware
interval = 30
timeout = 30
unhealthy_threshold = 3
protocol = "Https"
port = 443
path = "/actuator/health"
pick_host_name_from_backend_http_settings = true
match {
body = "UP"
status_code = [200]
}
}

# ------- Listeners -------------------------
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
Expand All @@ -116,11 +153,19 @@ resource "azurerm_application_gateway" "load_balancer" {
}

http_listener {
name = local.listener_name_api
name = local.listener_name_api_ocr
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name_static
protocol = "Http"
host_names = [var.fqdns_ocr]
}

http_listener {
name = local.listener_name_api_middleware
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name_static
protocol = "Http"
host_names = [var.fqdns]
host_names = [var.fqdns_middleware]
}

http_listener {
Expand All @@ -142,33 +187,48 @@ resource "azurerm_application_gateway" "load_balancer" {
}

request_routing_rule {
name = local.request_routing_rule_name_api
name = local.request_routing_rule_name_api_ocr
priority = 100
rule_type = "Basic"
http_listener_name = local.listener_name_api
backend_address_pool_name = local.backend_address_pool_name_api
backend_http_settings_name = local.http_setting_name_api
http_listener_name = local.listener_name_api_ocr
backend_address_pool_name = local.backend_address_pool_name_api_ocr
backend_http_settings_name = local.http_setting_name_api_ocr
}

request_routing_rule {
name = local.request_routing_rule_name_api_middleware
priority = 150
rule_type = "Basic"
http_listener_name = local.listener_name_api_middleware
backend_address_pool_name = local.backend_address_pool_name_api_middleware
backend_http_settings_name = local.http_setting_name_api_middleware
}

url_path_map {
name = "${var.name}-${var.env}-urlmap"
default_backend_address_pool_name = local.backend_address_pool_name_static
default_backend_http_settings_name = local.http_setting_name_static
default_rewrite_rule_set_name = "${var.name}-routing"
default_rewrite_rule_set_name = "${var.name}-middleware-routing"

path_rule {
name = "api"
name = "ocr"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the name just ocr or api_ocr?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh the name here doesn't really matter, just needs to be different than the middleware rule.

paths = ["/ocr-api/*", "/ocr-api"]
backend_address_pool_name = local.backend_address_pool_name_api
backend_http_settings_name = local.http_setting_name_api
backend_address_pool_name = local.backend_address_pool_name_api_ocr
backend_http_settings_name = local.http_setting_name_api_ocr
// this is the default, why would we set it again?
// because if we don't do this we get 404s on API calls
rewrite_rule_set_name = "${var.name}-routing"
rewrite_rule_set_name = "${var.name}-ocr-routing"
}
path_rule {
name = "middleware"
paths = ["/middleware-api/*", "/middleware-api"]
backend_address_pool_name = local.backend_address_pool_name_api_middleware
backend_http_settings_name = local.http_setting_name_api_middleware
rewrite_rule_set_name = "${var.name}-middleware-routing"
}
}
rewrite_rule_set {
name = "${var.name}-routing"
name = "${var.name}-ocr-routing"

rewrite_rule {
name = "ocr-api-wildcard"
Expand All @@ -189,4 +249,32 @@ resource "azurerm_application_gateway" "load_balancer" {
}
}
}

rewrite_rule_set {
name = "${var.name}-middleware-routing"

rewrite_rule {
name = "middleware-api-wildcard"
rule_sequence = 101
condition {
ignore_case = true
negate = false
pattern = ".*middleware-api/(.*)"
variable = "var_uri_path"
}

url {
path = "/{var_uri_path_1}"
reroute = false
# Per documentation, we should be able to leave this pass-through out. See however
# https://github.com/terraform-providers/terraform-provider-azurerm/issues/11563
query_string = "{var_query_string}"
}
}
}

autoscale_configuration {
min_capacity = 0
max_capacity = 5
}
}
9 changes: 0 additions & 9 deletions ops/terraform/modules/app_gateway/outputs.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,3 @@
output "fqdn" {
value = azurerm_public_ip.lb-pip.fqdn
}

output "app_gateway_hostname" {
value = azurerm_application_gateway.load_balancer.id
sensitive = true
}

output "app_gateway_ip" {
value = azurerm_public_ip.lb-pip.ip_address
}
4 changes: 3 additions & 1 deletion ops/terraform/modules/app_gateway/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ variable "zones" {
default = ["1", "2", "3"]
}

variable "fqdns" {
variable "fqdns_middleware" {
}
variable "fqdns_ocr" {
}

variable "ip_addresses" {
Expand Down
Loading
Loading