Add terraform-aws-proxy module

Signed-off-by: Jim Enright <[email protected]>

modules/terraform-aws-proxy/.terraform-docs.yaml

@@ -0,0 +1,21 @@
+formatter: markdown
+header-from: doc_fragments/header.md
+settings:
+  anchor: true
+  color: true
+  default: true
+  escape: true
+  html: true
+  indent: 2
+  required: true
+  sensitive: true
+  type: true
+
+
+sort:
+  enabled: true
+  by: required
+
+output:
+  file: README.md
+  mode: replace

modules/terraform-aws-proxy/README.md

@@ -0,0 +1,88 @@
+<!-- BEGIN_TF_DOCS -->
+# Terraform Module for AWS Transit Gateway
+
+This module contains resource files and example variable definition files for creation of AWS Transity Gateway (TGW) and attaching a specified list of VPCs via the TGW. This module also updates both the Transit Gateway and VPC route tables. This module can be used to assist in deploying Cloudera Data Platform (CDP) Public Cloud in a fully private networking configuration where a CDP VPC and Networking VPC are connected using the Transit Gateway.
+
+## Usage
+
+The [examples](./examples) directory has example of using this module:
+
+* `ex01-vpc-tgw-attach` demonstrates how this module can be used to use a Transit Gateway to attach a private CDP VPC with a dedicated networking VPC. The [terraform-aws-vpc](../../../terraform-aws-vpc/README.md) module is also used as part of this example.
+
+The README and sample `terraform.tfvars.sample` describe how to use the example.
+
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | > 1.3.0 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 4.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_aws"></a> [aws](#provider\_aws) | ~> 4.0 |
+
+## Modules
+
+No modules.
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [aws_autoscaling_attachment.proxy_asg_tg_attach](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_attachment) | resource |
+| [aws_autoscaling_group.proxy_asg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group) | resource |
+| [aws_launch_template.proxy_lt](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template) | resource |
+| [aws_lb.proxy_lb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb) | resource |
+| [aws_lb_listener.proxy_lb_listener](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener) | resource |
+| [aws_lb_target_group.proxy_tg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_target_group) | resource |
+| [aws_route.vpc_tgw_route](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) | resource |
+| [aws_security_group.proxy_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
+| [aws_security_group_rule.proxy_egress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
+| [aws_security_group_rule.proxy_ingress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
+| [aws_security_group_rule.proxy_lb_ingress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
+| [aws_ami.proxy_default_ami](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) | data source |
+| [aws_network_interface.proxy_lb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/network_interface) | data source |
+| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
+| [aws_route_table.proxy_rt](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route_table) | data source |
+| [aws_vpc.proxy_vpc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_lb_subnet_ids"></a> [lb\_subnet\_ids](#input\_lb\_subnet\_ids) | The IDs of the subnet for the Network Load Balancer | `list(any)` | n/a | yes |
+| <a name="input_network_load_balancer_name"></a> [network\_load\_balancer\_name](#input\_network\_load\_balancer\_name) | Name of Network Load Balancer for the Proxy. | `string` | n/a | yes |
+| <a name="input_proxy_autoscaling_group_name"></a> [proxy\_autoscaling\_group\_name](#input\_proxy\_autoscaling\_group\_name) | Name of Autoscaling Group for the Proxy VMs. | `string` | n/a | yes |
+| <a name="input_proxy_aws_keypair_name"></a> [proxy\_aws\_keypair\_name](#input\_proxy\_aws\_keypair\_name) | SSH Keypair name for the proxy VM | `string` | n/a | yes |
+| <a name="input_proxy_launch_template_name"></a> [proxy\_launch\_template\_name](#input\_proxy\_launch\_template\_name) | Name of Launch Template for the Proxy VMs. | `string` | n/a | yes |
+| <a name="input_proxy_subnet_ids"></a> [proxy\_subnet\_ids](#input\_proxy\_subnet\_ids) | The IDs of the subnet where the proxy VMs will run | `list(any)` | n/a | yes |
+| <a name="input_target_group_proxy_name"></a> [target\_group\_proxy\_name](#input\_target\_group\_proxy\_name) | Name of Target Group for the Proxy. | `string` | n/a | yes |
+| <a name="input_vpc_id"></a> [vpc\_id](#input\_vpc\_id) | VPC ID for where the proxy VM will run | `string` | n/a | yes |
+| <a name="input_autoscaling_group_scaling"></a> [autoscaling\_group\_scaling](#input\_autoscaling\_group\_scaling) | Minimum, maximum and desired size of EC2 instance in the Auto Scaling Group. | <pre>object({<br>    min_size         = number<br>    max_size         = number<br>    desired_capacity = number<br>  })</pre> | <pre>{<br>  "desired_capacity": 3,<br>  "max_size": 6,<br>  "min_size": 3<br>}</pre> | no |
+| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS region, used in Proxy Whitelist configuration files. If not provided will perform lookup of aws\_region data source. | `string` | `null` | no |
+| <a name="input_cdp_region"></a> [cdp\_region](#input\_cdp\_region) | CDP Control Plane region, used in Proxy Whitelist configuration files. | `string` | `"us-west-1"` | no |
+| <a name="input_create_proxy_sg"></a> [create\_proxy\_sg](#input\_create\_proxy\_sg) | Flag to specify if the Security Group for the proxy should be created. | `bool` | `true` | no |
+| <a name="input_egress_rules"></a> [egress\_rules](#input\_egress\_rules) | List of egress rules to create. Used only if create\_proxy\_sg is true | <pre>list(object({<br>    cidrs     = list(string)<br>    from_port = number<br>    to_port   = optional(number)<br>    protocol  = string<br>  }))</pre> | <pre>[<br>  {<br>    "cidrs": [<br>      "0.0.0.0/0"<br>    ],<br>    "from_port": 0,<br>    "protocol": "all",<br>    "to_port": 0<br>  }<br>]</pre> | no |
+| <a name="input_enable_proxy_public_ip"></a> [enable\_proxy\_public\_ip](#input\_enable\_proxy\_public\_ip) | Assign a public IP address to the Proxy VM | `bool` | `true` | no |
+| <a name="input_env_tags"></a> [env\_tags](#input\_env\_tags) | Tags applied to provisioned resources | `map(any)` | `{}` | no |
+| <a name="input_ingress_rules"></a> [ingress\_rules](#input\_ingress\_rules) | List of ingress rules to create. Used only if create\_proxy\_sg is true | <pre>list(object({<br>    cidrs     = list(string)<br>    from_port = number<br>    to_port   = optional(number)<br>    protocol  = string<br>  }))</pre> | `[]` | no |
+| <a name="input_proxy_aws_ami"></a> [proxy\_aws\_ami](#input\_proxy\_aws\_ami) | The AWS AMI to use for the proxy VM | `string` | `null` | no |
+| <a name="input_proxy_aws_instance_type"></a> [proxy\_aws\_instance\_type](#input\_proxy\_aws\_instance\_type) | The EC2 instance type to use for the proxy VM | `string` | `"t3.medium"` | no |
+| <a name="input_proxy_launch_template_user_data_file"></a> [proxy\_launch\_template\_user\_data\_file](#input\_proxy\_launch\_template\_user\_data\_file) | Location of the AWS Launch Template user data script. If not specified the files/user-data-proxy.sh.tpl file accompanying the module is used. | `string` | `null` | no |
+| <a name="input_proxy_port"></a> [proxy\_port](#input\_proxy\_port) | Port number which the proxy and NLB listens | `number` | `3129` | no |
+| <a name="input_proxy_security_group_id"></a> [proxy\_security\_group\_id](#input\_proxy\_security\_group\_id) | ID for existing Security Group to be used for the proxy VM. Required when create\_proxy\_sg is false | `string` | `null` | no |
+| <a name="input_proxy_security_group_name"></a> [proxy\_security\_group\_name](#input\_proxy\_security\_group\_name) | Name of Proxy Security Group for CDP environment. Used only if create\_proxy\_sg is true. | `string` | `null` | no |
+| <a name="input_proxy_whitelist_file"></a> [proxy\_whitelist\_file](#input\_proxy\_whitelist\_file) | Location of the Proxy Whitelist file. If not specified the files/squid-http-whitelist.txt.tpl file accompanying the module is used. | `string` | `null` | no |
+| <a name="input_route_tables_to_update"></a> [route\_tables\_to\_update](#input\_route\_tables\_to\_update) | List of any route tables to update to point to the Network interface of the Proxy VM | <pre>list(object({<br>    route_tables           = list(string)<br>    destination_cidr_block = string<br>  }))</pre> | `[]` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_proxy_lb_arn"></a> [proxy\_lb\_arn](#output\_proxy\_lb\_arn) | ARN of the Proxy Load Balancer |
+| <a name="output_proxy_lb_dns_name"></a> [proxy\_lb\_dns\_name](#output\_proxy\_lb\_dns\_name) | DNS Name of the Proxy Load Balancer |
+| <a name="output_proxy_port"></a> [proxy\_port](#output\_proxy\_port) | Port where Proxy is running |
+<!-- END_TF_DOCS -->

modules/terraform-aws-proxy/data.tf

@@ -0,0 +1,61 @@
+# Copyright 2023 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Find AWS region
+data "aws_region" "current" {}
+
+# Find AMI for default proxy VMs
+data "aws_ami" "proxy_default_ami" {
+
+  most_recent = true
+
+  filter {
+    name   = "name"
+    values = ["al2023-ami-2023*-x86_64"]
+  }
+
+  owners = ["amazon"]
+}
+
+# Find details of the VPC
+data "aws_vpc" "proxy_vpc" {
+  id = var.vpc_id
+}
+
+# Find the network interface for the load balancer
+data "aws_network_interface" "proxy_lb" {
+
+  for_each = { for k, v in var.lb_subnet_ids : k => v }
+
+  filter {
+    name   = "description"
+    values = ["ELB ${aws_lb.proxy_lb.arn_suffix}"]
+  }
+
+  filter {
+    name   = "subnet-id"
+    values = [each.value]
+  }
+}
+
+# Find route table details
+data "aws_route_table" "proxy_rt" {
+
+  for_each = {
+    for k, v in local.route_tables_to_update : k => v
+  }
+
+  route_table_id = each.value.route_table
+
+}

modules/terraform-aws-proxy/defaults.tf

@@ -0,0 +1,71 @@
+# Copyright 2023 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+locals {
+
+  # AWS region (derived from provider lookup unless overridden)
+  aws_region = coalesce(var.aws_region, data.aws_region.current.name)
+
+  # Security Groups
+  proxy_security_group_id = (var.create_proxy_sg ?
+  aws_security_group.proxy_sg[0].id : var.proxy_security_group_id)
+
+  # Proxy VM
+  proxy_aws_ami = coalesce(var.proxy_aws_ami, data.aws_ami.proxy_default_ami.id)
+
+  # User data for Proxy VM (for squid proxy)
+  proxy_launch_template_user_data_file = coalesce(var.proxy_launch_template_user_data_file, "${path.module}/files/squid-user-data.sh.tpl")
+
+  # Squid whitelist file
+  proxy_whitelist_file = coalesce(var.proxy_whitelist_file, "${path.module}/files/squid-whitelist.txt.tpl")
+
+  # Local variables to determine route table to Internal NLB eni mapping
+  route_tables_to_update = flatten([
+    for route in var.route_tables_to_update :
+    [
+      for rt in route.route_tables :
+      {
+        route_table            = rt
+        destination_cidr_block = route.destination_cidr_block
+      }
+    ]
+  ])
+
+  lb_eni_details = [
+    for eni in data.aws_network_interface.proxy_lb :
+    {
+      eni_id    = eni.id
+      az        = eni.availability_zone
+      subnet_id = eni.subnet_id
+    }
+  ]
+
+  # TODO: Explore better rt to eni mapping with the below
+  # route_table_details = [
+  #   for rt in data.aws_route_table.proxy_rt :
+  #   {
+  #     rt_id      = rt.id
+  #     subnet_ids = rt.associations[*].subnet_id
+  #   }
+  # ]
+
+  route_table_to_lb_eni_assoc = {
+    for k, v in data.aws_route_table.proxy_rt : v.id => {
+      # TODO: eni of same subnet assoc if possible otherwise the first eni_id in lb_eni_details
+      eni = local.lb_eni_details[0].eni_id
+    }
+  }
+
+
+}

modules/terraform-aws-proxy/doc_fragments/header.md

@@ -0,0 +1,11 @@
+# Terraform Module for AWS Transit Gateway
+
+This module contains resource files and example variable definition files for creation of AWS Transity Gateway (TGW) and attaching a specified list of VPCs via the TGW. This module also updates both the Transit Gateway and VPC route tables. This module can be used to assist in deploying Cloudera Data Platform (CDP) Public Cloud in a fully private networking configuration where a CDP VPC and Networking VPC are connected using the Transit Gateway.
+
+## Usage
+
+The [examples](./examples) directory has example of using this module:
+
+* `ex01-vpc-tgw-attach` demonstrates how this module can be used to use a Transit Gateway to attach a private CDP VPC with a dedicated networking VPC. The [terraform-aws-vpc](../../../terraform-aws-vpc/README.md) module is also used as part of this example.
+
+The README and sample `terraform.tfvars.sample` describe how to use the example.

modules/terraform-aws-proxy/examples/ex01-minimal_inputs/main.tf

@@ -0,0 +1,72 @@
+# Copyright 2023 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+provider "aws" {
+  profile = var.aws_profile
+  region  = var.aws_region
+}
+
+module "ex01_network_vpc" {
+  source = "../../../terraform-aws-vpc"
+
+  cdp_vpc            = false
+  vpc_name           = "${var.name_prefix}-network-vpc"
+  vpc_cidr           = "10.11.0.0/16"
+  enable_nat_gateway = false
+
+  private_cidr_range = var.network_vpc_private_cidr_range
+  public_cidr_range  = var.network_vpc_public_cidr_range
+
+}
+
+module "ex01_proxy" {
+  source = "../.."
+
+  vpc_id = module.ex01_network_vpc.vpc_id
+
+  proxy_security_group_name = "${var.name_prefix}-sg"
+  proxy_aws_keypair_name    = var.aws_key_pair
+
+  proxy_launch_template_name   = "${var.name_prefix}-lt"
+  proxy_autoscaling_group_name = "${var.name_prefix}-asg"
+  proxy_subnet_ids             = module.ex01_network_vpc.public_subnets
+
+  network_load_balancer_name = "${var.name_prefix}-lb"
+  target_group_proxy_name    = "${var.name_prefix}-tg"
+  lb_subnet_ids              = module.ex01_network_vpc.private_subnets
+
+  ingress_rules = [
+    {
+      cidrs     = module.ex01_network_vpc.vpc_cidr_blocks
+      from_port = 0
+      to_port   = 65535
+      protocol  = "tcp"
+    },
+    {
+      cidrs     = var.ingress_extra_cidrs
+      from_port = 22
+      # to_port = 
+      protocol = "tcp"
+    }
+  ]
+
+  route_tables_to_update = [
+    # Route all Internet traffic in Networking VPC to the proxy instance(s)
+    {
+      route_tables           = module.ex01_network_vpc.private_route_tables
+      destination_cidr_block = "0.0.0.0/0"
+    }
+  ]
+
+}

modules/terraform-aws-proxy/examples/ex01-minimal_inputs/terraform.tfvars.sample

@@ -0,0 +1,23 @@
+# Copyright 2023 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# ------- Global Settings -------
+name_prefix = "<ENTER_VALUE>"
+
+# ------- Cloud Settings -------
+aws_region = "<ENTER_VALUE>" # Change this to specify Cloud Provider region, e.g. eu-west-1
+aws_key_pair = "<ENTER_VALUE>" # Change this with the name of a pre-existing AWS keypair, e.g. my-keypair
+
+# ------- Proxy settings -------
+ingress_extra_cidrs = "<ENTER_VALUE>" # Any additional CIDRs the Proxy Security Groups for SSH access