Skip to content

Latest commit

 

History

History
176 lines (159 loc) · 26.9 KB

README.md

File metadata and controls

176 lines (159 loc) · 26.9 KB
Raw

Terraform Module for CDP Prerequisites on Azure

This module contains resource files and example variable definition files for creation of the pre-requisite Azure cloud resources required for Cloudera Data Platform (CDP) Public Cloud.

Usage

The examples directory has example Azure Cloud Service Provider deployments for different scenarios:

  • ex01-minimal-inputs uses the minimum set of inputs for the module.

In each directory an example terraform.tfvars.sample values file is included to show input variable values.

Requirements

Name Version
terraform >= 1.3.0
azuread 2.46.0
azurerm >= 4.0.0
random ~> 3.4.3

Providers

Name Version
azuread 2.46.0
azurerm >= 4.0.0
random ~> 3.4.3

Modules

Name Source Version
azure_cdp_vnet ./modules/vnet n/a
azure_cml_nfs ../terraform-azure-nfs n/a
stor_private_endpoints ../terraform-azure-endpoints n/a

Resources

Name Type
azuread_application.cdp_xaccount_app resource
azuread_application_password.cdp_xaccount_app_password resource
azuread_service_principal.cdp_xaccount_app_sp resource
azurerm_network_security_group.cdp_default_sg resource
azurerm_network_security_group.cdp_knox_sg resource
azurerm_network_security_rule.cdp_default_sg_ingress_extra_access resource
azurerm_network_security_rule.cdp_knox_sg_ingress_extra_access resource
azurerm_private_dns_zone.flexible_server_dns_zone resource
azurerm_private_dns_zone_virtual_network_link.flexible_server_vnet_link resource
azurerm_resource_group.cdp_rmgp resource
azurerm_role_assignment.cdp_datalake_admin_backup_container_assign resource
azurerm_role_assignment.cdp_datalake_admin_data_container_assign resource
azurerm_role_assignment.cdp_datalake_admin_log_container_assign resource
azurerm_role_assignment.cdp_idbroker_assign resource
azurerm_role_assignment.cdp_log_data_access_backup_container_assign resource
azurerm_role_assignment.cdp_log_data_access_log_container_assign resource
azurerm_role_assignment.cdp_ranger_audit_backup_container_assign resource
azurerm_role_assignment.cdp_ranger_audit_data_container_assign resource
azurerm_role_assignment.cdp_ranger_audit_log_container_assign resource
azurerm_role_assignment.cdp_raz_assign resource
azurerm_role_assignment.cdp_xaccount_role resource
azurerm_storage_account.cdp_storage_locations resource
azurerm_storage_account_network_rules.cdp_storage_access_rules resource
azurerm_storage_container.cdp_backup_storage resource
azurerm_storage_container.cdp_data_storage resource
azurerm_storage_container.cdp_log_storage resource
azurerm_user_assigned_identity.cdp_datalake_admin resource
azurerm_user_assigned_identity.cdp_idbroker resource
azurerm_user_assigned_identity.cdp_log_data_access resource
azurerm_user_assigned_identity.cdp_ranger_audit_data_access resource
azurerm_user_assigned_identity.cdp_raz resource
random_id.bucket_suffix resource
azuread_client_config.current data source
azurerm_resource_group.cdp_rmgp data source
azurerm_subnet.cdp_subnets data source
azurerm_subscription.current data source
azurerm_virtual_network.cdp_vnet data source

Inputs

Name Description Type Default Required
deployment_template Deployment Pattern to use for Cloud resources and CDP string n/a yes
env_prefix Shorthand name for the environment. Used in resource descriptions string n/a yes
agent_source_tag Tag to identify deployment source map(any) 
{
  "agent_source": "tf-cdp-module"
}
 no
azure_region Region which Cloud resources will be created string null no
backup_storage Optional Backup location for CDP environment. If not provided follow the data_storage variable 
object({
    backup_storage_bucket = string
    backup_storage_object = string
  })
 null no
cdp_delegated_subnet_names List of subnet names delegated for Flexible Servers. Required if create_vnet is false. list(any) null no
cdp_gw_subnet_names List of subnet names for CDP Gateway. Required if create_vnet is false. list(any) null no
cdp_resourcegroup_name Pre-existing Resource Group for CDP environment. Required if create_vnet is false. string null no
cdp_subnet_names List of subnet names for CDP Resources. Required if create_vnet is false. list(any) null no
cdp_subnet_range Size of each (internal) cluster subnet. Required if create_vpc is true. number 19 no
cdp_subnets_private_endpoint_network_policies Enable or Disable network policies for the private endpoint on the CDP subnets string "Enabled" no
cdp_vnet_name Pre-existing VNet Name for CDP environment. Required if create_vnet is false. string null no
create_azure_cml_nfs Whether to create NFS for CML bool false no
create_azure_storage_network_rules Enable creation of network rules for the Azure Storage Accounts. bool false no
create_azure_storage_private_endpoints Flag to specify if Private Endpoints are created for each storage account. bool true no
create_private_flexible_server_resources Flag to specify if resources to support a Private Postgres flexible server should be created. bool null no
create_vm_mounting_nfs Whether to create a VM which mounts this NFS bool true no
create_vnet Flag to specify if the VNet should be created bool true no
data_storage Data storage locations for CDP environment 
object({
    data_storage_bucket = string
    data_storage_object = string
  })
 null no
datalake_admin_backup_container_role_assignments List of Role Assignments for the Datalake Admin Managed Identity assigned to the Backup Storage Container. 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign Storage Blob Data Owner Role to Data Lake Admin Identity at Backup Container Level",
    "role": "Storage Blob Data Owner"
  }
]
 no
datalake_admin_data_container_role_assignments List of Role Assignments for the Datalake Admin Managed Identity assigned to the Data Storage Container. 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign Storage Blob Data Owner Role to Data Lake Admin Identity at Data Container Level",
    "role": "Storage Blob Data Owner"
  }
]
 no
datalake_admin_log_container_role_assignments List of Role Assignments for the Datalake Admin Managed Identity assigned to the Logs Storage Container. 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign Storage Blob Data Owner Role to Data Lake Admin Identity at Logs Container Level",
    "role": "Storage Blob Data Owner"
  }
]
 no
datalake_admin_managed_identity_name Datalake Admin Managed Identity name string null no
delegated_subnet_range Size of each Postgres Flexible Server delegated subnet. Required if create_vpc is true. number 26 no
enable_raz Flag to enable Ranger Authorization Service (RAZ) bool true no
env_tags Tags applied to provisioned resources map(any) null no
gateway_subnet_range Size of each gateway subnet. Required if create_vpc is true. number 24 no
gateway_subnets_private_endpoint_network_policies Enable or Disable network policies for the private endpoint on the Gateway subnets string "Enabled" no
idbroker_managed_identity_name IDBroker Managed Identity name string null no
idbroker_role_assignments List of Role Assignments for the IDBroker Managed Identity 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign VM Contributor Role to IDBroker Identity at Subscription Level",
    "role": "Virtual Machine Contributor"
  },
  {
    "description": "Assign Managed Identity Operator Role to IDBroker Identity at Subscription Level",
    "role": "Managed Identity Operator"
  }
]
 no
ingress_extra_cidrs_and_ports List of extra CIDR blocks and ports to include in Security Group Ingress rules 
object({
    cidrs = list(string)
    ports = list(number)
  })
 
{
  "cidrs": [],
  "ports": []
}
 no
log_data_access_managed_identity_name Log Data Access Managed Identity name string null no
log_data_access_role_assignments List of Role Assignments for the Log Data Access Managed Identity. 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign Storage Blob Data Contributor Role to Log Role at Logs and Backup Container level",
    "role": "Storage Blob Data Contributor"
  }
]
 no
log_storage Optional log locations for CDP environment. If not provided follow the data_storage variable 
object({
    log_storage_bucket = string
    log_storage_object = string
  })
 null no
nfs_file_share_name nfs file share name string null no
nfs_file_share_size NFS File Share size number 100 no
nfs_storage_account_name NFS Storage account name string null no
public_key_text SSH Public key string for the nodes of the CDP environment string null no
random_id_for_bucket Create a random suffix for the Storage Account names bool true no
ranger_audit_backup_container_role_assignments List of Role Assignments for the Ranger Audit Managed Identity assigned to the Backup Storage Container. 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign Storage Blob Data Contributor Role to Ranger Audit Role at Backup Container level",
    "role": "Storage Blob Data Contributor"
  }
]
 no
ranger_audit_data_access_managed_identity_name Ranger Audit Managed Identity name string null no
ranger_audit_data_container_role_assignments List of Role Assignments for the Ranger Audit Managed Identity assigned to the Data Storage Container. 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign Storage Blob Data Contributor Role to Ranger Audit Role at Data Container level",
    "role": "Storage Blob Data Contributor"
  }
]
 no
ranger_audit_log_container_role_assignments List of Role Assignments for the Ranger Audit Managed Identity assigned to the Log Storage Container. 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign Storage Blob Data Contributor Role to Ranger Audit Role at Logs Container level",
    "role": "Storage Blob Data Contributor"
  }
]
 no
raz_managed_identity_name RAZ Managed Identity name string null no
raz_storage_role_assignments List of Role Assignments for the Ranger Audit Managed Identity assigned to the Log Storage Container. 
list(object({
    role        = string
    description = string
    })
  )
 
[
  {
    "description": "Assign Storage Blob Delegator Role to RAZ Identity at Storage Account level",
    "role": "Storage Blob Delegator"
  },
  {
    "description": "Assign Storage Blob Data Owner Role to RAZ Identity at Storage Account level",
    "role": "Storage Blob Data Owner"
  }
]
 no
resourcegroup_name Resource Group name string null no
security_group_default_name Default Security Group for CDP environment string null no
security_group_knox_name Knox Security Group for CDP environment string null no
storage_public_network_access_enabled Enable public_network_access_enabled for storage accounts. bool true no
subnet_count Number of CDP Subnets Required string "3" no
vnet_cidr VNet CIDR Block. Required if create_vpc is true. string "10.10.0.0/16" no
vnet_name VNet name string null no
xaccount_app_name Cross account application name within Azure Active Directory string null no

Outputs

Name Description
azure_backup_storage_account Azure backup storage account name
azure_backup_storage_container Azure backup storage container name
azure_backup_storage_location Azure backup storage location
azure_cdp_flexible_server_delegated_subnet_names Azure Virtual Subnet Names delegated for Private Flexible servers.
azure_cdp_gateway_subnet_names Azure Virtual Subnet Names for CDP Endpoint Access Gateway
azure_cdp_subnet_names Azure Virtual Subnet Names for CDP Resources
azure_data_storage_account Azure data storage account name
azure_data_storage_container Azure data storage container name
azure_data_storage_location Azure data storage location
azure_database_private_dns_zone_id The ID of an Azure private DNS zone used for the database.
azure_datalakeadmin_identity_id Datalake Admin Managed Identity ID
azure_idbroker_identity_id IDBroker Managed Identity ID
azure_log_identity_id Log Data Access Managed Identity ID
azure_log_storage_account Azure log storage account name
azure_log_storage_container Azure log storage container name
azure_log_storage_location Azure log storage location
azure_ranger_audit_identity_id Ranger Audit Managed Identity ID
azure_raz_identity_id RAZ Managed Identity ID. Value returned if RAZ is enabled
azure_resource_group_name Azure Resource Group Name
azure_security_group_default_uri Azure Default Security Group URI
azure_security_group_knox_uri Azure Knox Security Group URI
azure_subscription_id Subscription ID where the Azure pre-reqs are created
azure_tenant_id Tenant ID where the Azure pre-reqs are created
azure_vnet_adress_space Azure Virtual Network Address Space
azure_vnet_id Azure Virtual Network ID
azure_vnet_name Azure Virtual Network Name
azure_xaccount_app_pword Password for the Azure AD Cross Account Application
azure_xaccount_app_uuid UUID for the Azure AD Cross Account Application
nfs_file_share_url NFS File Share Url
nfs_storage_account_name NFS Storage Account Name
nfs_vm_mount_path Path where NFS is mounted on the VM
nfs_vm_public_ip NFS VM Public IP
nfs_vm_username NFS VM Admin Username