Skip to content

Latest commit

 

History

History
131 lines (114 loc) · 17 KB

File metadata and controls

131 lines (114 loc) · 17 KB

Terraform Module for CDP Prerequisites on GCP

This module contains resource files and example variable definition files for creation of the pre-requisite Google Cloud Platform (GCP) resources required for Cloudera Data Platform (CDP) Public Cloud.

Usage

The examples directory has example GCP Cloud Service Provider deployments for different scenarios:

  • ex01-minimal-inputs uses the minimum set of inputs for the module.

In each directory an example terraform.tfvars.sample values file is included to show input variable values.

Requirements

Name Version
terraform > 1.3.0
google >= 6.12
random ~> 3.4.3

Providers

Name Version
google >= 6.12
random ~> 3.4.3

Modules

Name Source Version
gcp_cdp_vpc ./modules/vpc n/a

Resources

Name Type
google_compute_firewall.cdp_allow_internal_fw resource
google_compute_firewall.cdp_default_fw resource
google_compute_firewall.cdp_knox_fw resource
google_compute_global_address.google_managed_services resource
google_compute_router.cdp_compute_router resource
google_compute_router_nat.cdp_nat resource
google_project_iam_custom_role.cdp_datalake_admin resource
google_project_iam_custom_role.cdp_idbroker resource
google_project_iam_custom_role.cdp_log_data_access resource
google_project_iam_member.cdp_datalake_admin_sa_binding resource
google_project_iam_member.cdp_idbroker_sa_binding resource
google_project_iam_member.cdp_log_sa_binding resource
google_project_iam_member.cdp_ranger_audit_sa_binding resource
google_project_iam_member.cdp_xaccount_sa_binding resource
google_service_account.cdp_datalake_admin_sa resource
google_service_account.cdp_idbroker_sa resource
google_service_account.cdp_log_sa resource
google_service_account.cdp_ranger_audit_sa resource
google_service_account.cdp_xaccount_sa resource
google_service_account_iam_member.cdp_idbroker_dladmin_binding resource
google_service_account_iam_member.cdp_idbroker_ranger_audit_binding resource
google_service_account_key.cdp_xaccount_sa_key resource
google_service_networking_connection.google_managed_services resource
google_storage_bucket.cdp_storage_locations resource
google_storage_bucket_iam_member.cdp_data_sa_member resource
google_storage_bucket_iam_member.cdp_log_sa_member resource
google_storage_bucket_iam_member.cdp_ranger_audit_sa_member resource
random_id.bucket_suffix resource
google_project.project data source

Inputs

Name Description Type Default Required
deployment_template Deployment Pattern to use for Cloud resources and CDP string n/a yes
env_prefix Shorthand name for the environment. Used in resource descriptions string n/a yes
backup_storage_bucket Optional Backup location for CDP environment. string null no
bucket_storage_class The GCS storage class to use for the data, log and backup storage string "NEARLINE" no
bucket_storage_region The location of the Google Cloud Storage buckets for data, backups and logs. By default this follows the gcp_region variable. string null no
cdp_subnet_names List of subnet names. Required if create_vpc is false. list(any) null no
cdp_vpc_name VPC Name for CDP environment. Required if create_vpc is false. string null no
compute_router_bgp_settings BGP settings used for the Google Compute Router resource in private deployments.
object({
asn = number
advertise_mode = optional(string)
advertised_groups = optional(string)
advertised_ip_ranges = optional(list(object({})))
keepalive_interval = optional(number)
})
{
"advertise_mode": "DEFAULT",
"asn": 64514
}
no
compute_router_name Name of the Google Compute Router resource created for private deployment. string null no
compute_router_nat_ip_allocate_option How external IPs should be allocated for Google Compute Router NAT in private deployments. string "AUTO_ONLY" no
compute_router_nat_name Name of the Google Compute Router NAT created for private deployment. string null no
compute_router_nat_source_subnetwork_ip_ranges How NAT should be configured per Subnetwork for Google Compute Router NAT in private deployments. string "ALL_SUBNETWORKS_ALL_IP_RANGES" no
create_vpc Flag to specify if the VPC Network should be created bool true no
data_storage_bucket Data storage locations for CDP environment string null no
datalake_admin_custom_role_name Name of Ranger Audit and Datalake Admin Custom Role string null no
datalake_admin_role_permissions List of Permission Assignments to the Ranger Audit and Datalake Admin Custom Role list(string)
[
"storage.buckets.get",
"storage.objects.create",
"storage.objects.delete",
"storage.objects.get",
"storage.objects.list",
"storage.hmacKeys.create",
"storage.hmacKeys.delete",
"storage.hmacKeys.get",
"storage.hmacKeys.list",
"storage.hmacKeys.update"
]
no
datalake_admin_service_account_name Datalake Admin service account name string null no
firewall_default_name Name of Default Firewall for CDP environment string null no
firewall_internal_name Name of Firewall for Internal Virtual Network communication string null no
firewall_knox_name Name of Knox Firewall for CDP environment string null no
gcp_region Region which Cloud resources will be created string null no
idbroker_custom_role_name Name of IDBroker Custom Role string null no
idbroker_role_permissions List of Permission Assignments to the IDBroker Custom Role list(string)
[
"iam.serviceAccounts.getAccessToken",
"iam.serviceAccounts.actAs"
]
no
idbroker_service_account_name IDBroker service account name string null no
ingress_extra_cidrs_and_ports List of extra CIDR blocks and ports to include in Security Group Ingress rules
object({
cidrs = list(string)
ports = list(number)
})
{
"cidrs": [],
"ports": []
}
no
log_data_access_custom_role_name Name of Log Data Access Custom Role string null no
log_role_permissions List of Permission Assignments to the Log Data Access Custom Role list(string)
[
"storage.buckets.get",
"storage.objects.create"
]
no
log_service_account_name Log service account name string null no
log_storage_bucket Optional log locations for CDP environment. string null no
managed_services_global_address_cidr CIDR Block for Google Managed Service VPC Peering Connection Address string "10.10.192.0/24" no
managed_services_global_address_name Name of the Managed Service address used for the Peering Connection to CloudSQL string null no
random_id_for_bucket Create a random suffix for the bucket names bool true no
ranger_audit_service_account_name Ranger Audit service account name string null no
subnet_count Number of Subnets Required number 1 no
vpc_cidr VPC CIDR Block string "10.1.0.0/19" no
vpc_name VPC name string null no
xaccount_sa_policies List of IAM policies to apply to the Cross Account Service Account list(string)
[
"roles/iam.serviceAccountUser",
"roles/compute.instanceAdmin.v1",
"roles/storage.admin",
"roles/compute.networkViewer",
"roles/compute.loadBalancerAdmin",
"roles/cloudsql.admin",
"roles/compute.networkUser",
"roles/compute.publicIpAdmin",
"roles/cloudkms.admin"
]
no
xaccount_service_account_name Cross Account service account name string null no

Outputs

Name Description
gcp_backup_storage_location GCP log storage location
gcp_cdp_subnet_names GCP VPC Subnet Names for CDP Resources
gcp_data_storage_location GCP data storage location
gcp_datalake_admin_service_account_email Email id of the service account for Datalake Admin
gcp_firewall_default_id GCP Default Firewall Rule ID
gcp_firewall_default_name GCP Default Firewall Rule Name
gcp_firewall_knox_id GCP Knox Firewall Rule ID
gcp_firewall_knox_name GCP Knox Firewall Rule Name
gcp_idbroker_service_account_email Email id of the service account for IDBroker
gcp_log_service_account_email Email id of the service account for Log Storage
gcp_log_storage_location GCP log storage location
gcp_ranger_audit_service_account_email Email id of the service account for Ranger Audit
gcp_vpc_name GCP VPC Network name
gcp_xaccount_sa_private_key Base64 encoded private key of the GCP Cross Account Service Account Key
gcp_xaccount_sa_public_key Base64 encoded public key of the GCP Cross Account Service Account Key