terraform-azurerm-avm-ptn-aks-production

NOTE: This module follows the semantic versioning and versions prior to 1.0.0 should be consider pre-release versions.

This is the Production Standard for AKS pattern module for Azure Verified Modules (AVM) library. This module deploys a production standard AKS cluster along with supporting a Virtual Network and Azure container registry. It provisions an environment sufficient for most production deployments for AKS. It leverages the AzureRM provider and sets a number of initial defaults to minimize the overall inputs for simple configurations.

Major version Zero (0.y.z) is for initial development. Anything MAY change at any time. A module SHOULD NOT be considered stable till at least it is major version one (1.0.0) or greater. Changes will always be via new versions being published and no changes will be made to existing published versions. For more details please go to https://semver.org/

Deployment Steps

1. Set up a GitHub repo environment called test.
2. Configure environment protection rule to ensure that approval is required before deploying to this environment.
3. Create a user-assigned managed identity in your test subscription.
4. Create a role assignment for the managed identity on your test subscription, use the minimum required role.
5. Configure federated identity credentials on the user assigned managed identity. Use the GitHub environment.
6. Create the following environment secrets on the test environment:
   1. AZURE_CLIENT_ID
   2. AZURE_TENANT_ID
   3. AZURE_SUBSCRIPTION_ID
7. Search and update TODOs within the code and remove the TODO comments once complete.

Requirements

The following requirements are needed by this module:

• terraform (>= 1.3.0)

• azapi (>= 1.4.0, < 2.0)

• azurerm (>= 3.86.0)

• local (2.4.1)

• null (>= 3.0)

• random (>= 3.5.0)

Providers

The following providers are used by this module:

• azapi (>= 1.4.0, < 2.0)

• azurerm (>= 3.86.0)

• local (2.4.1)

• null (>= 3.0)

• random (>= 3.5.0)

Resources

The following resources are used by this module:

• azapi_update_resource.aks_cluster_post_create (resource)
• azurerm_container_registry.this (resource)
• azurerm_kubernetes_cluster.this (resource)
• azurerm_kubernetes_cluster_node_pool.this (resource)
• azurerm_log_analytics_workspace.this (resource)
• azurerm_log_analytics_workspace_table.this (resource)
• azurerm_management_lock.this (resource)
• azurerm_monitor_diagnostic_setting.aks (resource)
• azurerm_resource_group_template_deployment.telemetry (resource)
• azurerm_role_assignment.acr (resource)
• azurerm_user_assigned_identity.aks (resource)
• null_resource.kubernetes_version_keeper (resource)
• random_id.telem (resource)
• random_string.acr_suffix (resource)
• local_file.compute_provider (data source)
• local_file.locations (data source)

Required Inputs

The following input variables are required:

location

Description: The Azure region where the resources should be deployed.

Type: string

name

Description: The name for the AKS resources created in the specified Azure Resource Group. This variable overwrites the 'prefix' var (The 'prefix' var will still be applied to the dns_prefix if it is set)

Type: string

resource_group_name

Description: The resource group where the resources will be deployed.

Type: string

Optional Inputs

The following input variables are optional (have default values):

agents_tags

Description: (Optional) A mapping of tags to assign to the Node Pool.

Type: map(string)

Default: {}

client_id

Description: (Optional) The Client ID (appId) for the Service Principal used for the AKS deployment

Type: string

Default: ""

client_secret

Description: (Optional) The Client Secret (password) for the Service Principal used for the AKS deployment

Type: string

Default: ""

enable_telemetry

Description: This variable controls whether or not telemetry is enabled for the module.
For more information see https://aka.ms/avm/telemetryinfo.
If it is set to false, then no telemetry will be collected.

Type: bool

Default: true

key_vault_secrets_provider_enabled

Description: (Optional) Whether to use the Azure Key Vault Provider for Secrets Store CSI Driver in an AKS cluster. For more details: https://docs.microsoft.com/en-us/azure/aks/csi-secrets-store-driver

Type: bool

Default: false

kubernetes_version

Description: Specify which Kubernetes release to use. Specify only minor version, such as '1.28'.

Type: string

Default: null

lock

Description: Controls the Resource Lock configuration for this resource. The following properties can be specified:

• kind - (Required) The type of lock. Possible values are \"CanNotDelete\" and \"ReadOnly\".
• name - (Optional) The name of the lock. If not specified, a name will be generated based on the kind value. Changing this forces the creation of a new resource.

Type:

object({
    kind = string
    name = optional(string, null)
  })

Default: null

log_analytics_workspace_id

Description: (Optional) The ID of the Log Analytics Workspace to use for the OMS agent.

Type: string

Default: null

managed_identities

Description: Controls the Managed Identity configuration on this resource. The following properties can be specified:

• system_assigned - (Optional) Specifies if the System Assigned Managed Identity should be enabled.
• user_assigned_resource_ids - (Optional) Specifies a list of User Assigned Managed Identity resource IDs to be assigned to this resource.

Type:

object({
    system_assigned            = optional(bool, false)
    user_assigned_resource_ids = optional(set(string), [])
  })

Default: {}

monitor_metrics

Description: (Optional) Specifies a Prometheus add-on profile for the Kubernetes Cluster
object({
annotations_allowed = "(Optional) Specifies a comma-separated list of Kubernetes annotation keys that will be used in the resource's labels metric."
labels_allowed = "(Optional) Specifies a Comma-separated list of additional Kubernetes label keys that will be used in the resource's labels metric." })

Type:

object({
    annotations_allowed = optional(string)
    labels_allowed      = optional(string)
  })

Default: null

node_cidr

Description: (Optional) The CIDR to use for node IPs in the Kubernetes cluster. Changing this forces a new resource to be created.

Type: string

Default: null

node_pools

Description: A map of node pools that need to be created and attached on the Kubernetes cluster. The key of the map can be the name of the node pool, and the key must be static string. The value of the map is a node_pool block as defined below:
map(object({
name = (Required) The name of the Node Pool which should be created within the Kubernetes Cluster. Changing this forces a new resource to be created. A Windows Node Pool cannot have a name longer than 6 characters. A random suffix of 4 characters is always added to the name to avoid clashes during recreates.
vm_size = (Required) The SKU which should be used for the Virtual Machines used in this Node Pool. Changing this forces a new resource to be created.
orchestrator_version = (Required) The version of Kubernetes which should be used for this Node Pool. Changing this forces a new resource to be created.
max_count = (Optional) The maximum number of nodes which should exist within this Node Pool. Valid values are between 0 and 1000 and must be greater than or equal to min_count.
min_count = (Optional) The minimum number of nodes which should exist within this Node Pool. Valid values are between 0 and 1000 and must be less than or equal to max_count.
os_sku = (Optional) Specifies the OS SKU used by the agent pool. Possible values include: Ubuntu, CBLMariner, Mariner, Windows2019, Windows2022. If not specified, the default is Ubuntu if OSType=Linux or Windows2019 if OSType=Windows. And the default Windows OSSKU will be changed to Windows2022 after Windows2019 is deprecated. Changing this forces a new resource to be created.
mode = (Optional) Should this Node Pool be used for System or User resources? Possible values are System and User. Defaults to User.
os_disk_size_gb = (Optional) The Agent Operating System disk size in GB. Changing this forces a new resource to be created.
tags = (Optional) A mapping of tags to assign to the resource. At this time there's a bug in the AKS API where Tags for a Node Pool are not stored in the correct case - you may wish to use Terraform's ignore_changes functionality to ignore changes to the casing until this is fixed in the AKS API.
zones = (Optional) Specifies a list of Availability Zones in which this Kubernetes Cluster Node Pool should be located. Changing this forces a new Kubernetes Cluster Node Pool to be created. }))

Example input:

  node_pools = {
    workload = {
      name                 = "workload"
      vm_size              = "Standard_D2d_v5"
      orchestrator_version = "1.28"
      max_count            = 110
      min_count            = 2
      os_sku               = "Ubuntu"
      mode                 = "User"
    },
    ingress = {
      name                 = "ingress"
      vm_size              = "Standard_D2d_v5"
      orchestrator_version = "1.28"
      max_count            = 4
      min_count            = 2
      os_sku               = "Ubuntu"
      mode                 = "User"
    }
  }

Type:

map(object({
    name                 = string
    vm_size              = string
    orchestrator_version = string
    # do not add nodecount because we enforce the use of auto-scaling
    max_count       = optional(number)
    min_count       = optional(number)
    os_sku          = optional(string)
    mode            = optional(string)
    os_disk_size_gb = optional(number, null)
    tags            = optional(map(string), {})
    zones           = optional(set(string))
  }))

Default: {}

orchestrator_version

Description: Specify which Kubernetes release to use. Specify only minor version, such as '1.28'.

Type: string

Default: null

pod_cidr

Description: (Optional) The CIDR to use for pod IPs in the Kubernetes cluster. Changing this forces a new resource to be created.

Type: string

Default: null

rbac_aad_admin_group_object_ids

Description: Object ID of groups with admin access.

Type: list(string)

Default: null

rbac_aad_azure_rbac_enabled

Description: (Optional) Is Role Based Access Control based on Azure AD enabled?

Type: bool

Default: null

rbac_aad_tenant_id

Description: (Optional) The Tenant ID used for Azure Active Directory Application. If this isn't specified the Tenant ID of the current Subscription is used.

Type: string

Default: null

tags

Description: (Optional) Tags of the resource.

Type: map(string)

Default: null

Outputs

The following outputs are exported:

resource

Description: This is the full output for the resource.

resource_id

Description: The azurerm_kubernetes_cluster's resource id.

Modules

The following Modules are called:

regions

Source: Azure/regions/azurerm

Version: >= 0.3.0

vnet

Source: Azure/subnets/azurerm

Version: 1.0.0

Data Collection

The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft’s privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.