AWS OpenSearch Terraform Module

Terraform module to provision an OpenSearch cluster with SAML authentication.

Prerequisites

• A hosted zone to route traffic to your OpenSearch domain
• An entityID and metadata XML from your SAML identity provider (in case saml_enabled = true)

Features

• Create an AWS OpenSearch cluster with SAML authentication
• All node types with local NVMe for high IO performance are supported
• Create or manage various OpenSearch resources:
  • Index templates
  • Indices
  • ISM policies
  • Roles
  • Role mappings

Usage

This example is using Azure AD as SAML identity provider.

locals {
  cluster_name      = "opensearch"
  cluster_domain    = "example.com"
  saml_entity_id    = "https://sts.windows.net/XXX-XXX-XXX-XXX-XXX/"
  saml_metadata_url = "https://login.microsoftonline.com/XXX-XXX-XXX-XXX-XXX/federationmetadata/2007-06/federationmetadata.xml?appid=YYY-YYY-YYY-YYY-YYY"
}

data "aws_region" "current" {}

data "http" "saml_metadata" {
  url = local.saml_metadata_url
}

provider "opensearch" {
  url                   = module.opensearch.cluster_endpoint
  aws_region            = data.aws_region.current.name
  healthcheck           = false
}

module "opensearch" {
  source  = "idealo/opensearch/aws"
  version = "~> 2.0"

  cluster_name    = local.cluster_name
  cluster_domain  = local.cluster_domain
  cluster_version = "2.11"

  saml_entity_id        = local.saml_entity_id
  saml_metadata_content = data.http.saml_metadata.body

  indices = {
    example-index-1 = {
      number_of_shards   = 2
      number_of_replicas = 1
    }
    example-index-2 = {
      number_of_shards   = 2
      number_of_replicas = 1
      mappings = {
        "properties" : {
          "id" : {
            "type" : "text"
          },
          "name" : {
            "type" : "text"
          },
          "containerType" : {
            "type" : "text"
          },
          "containerIds" : {
            "type" : "text"
          },
          "synonyms" : {
            "type" : "text"
          },
          "parentEvents" : {
            "type" : "text"
          },
          "valueType" : {
            "type" : "text"
          }
        }
      }
    }
  }
}

Examples

Here is a working example of using this Terraform module:

• Complete - Create an AWS OpenSearch cluster with all necessary resources.
• Minimal - Create an empty AWS OpenSearch cluster without saml.

Requirements

Name	Version
terraform	>= 1.3.0
aws	>= 4.12.0
opensearch	>= 2.0.0

Modules

Name	Source	Version
acm	terraform-aws-modules/acm/aws	~> 5.0.0

Resources

Name	Type
aws_cloudwatch_log_group.opensearch	resource
aws_cloudwatch_log_resource_policy.allow_logging	resource
aws_elasticsearch_domain.opensearch	resource
aws_elasticsearch_domain_saml_options.opensearch	resource
aws_iam_service_linked_role.es	resource
aws_route53_record.opensearch	resource
opensearch_composable_index_template.composable_index_template	resource
opensearch_index.index	resource
opensearch_index_template.index_template	resource
opensearch_ism_policy.ism_policy	resource
opensearch_role.role	resource
opensearch_roles_mapping.master_user_arn	resource
opensearch_roles_mapping.master_user_name	resource
opensearch_roles_mapping.role_mapping	resource
aws_caller_identity.current	data source
aws_iam_policy_document.access_policy	data source
aws_iam_policy_document.allow_logging	data source
aws_region.current	data source
aws_route53_zone.opensearch	data source

Inputs

Name	Description	Type	Default	Required
access_policies	IAM policy document specifying the access policies for the domain.	string	null	no
advanced_options	Key-value string pairs to specify advanced configuration options.	map(string)	null	no
advanced_security_options_enabled	Whether advanced security is enabled.	bool	true	no
advanced_security_options_internal_user_database_enabled	Whether to enable or not internal Kibana user database for ELK OpenDistro security plugin	bool	false	no
advanced_security_options_master_user_name	Master user username (applicable if advanced_security_options_internal_user_database_enabled set to true)	string	null	no
advanced_security_options_master_user_password	Master user password (applicable if advanced_security_options_internal_user_database_enabled set to true)	string	null	no
auto_tune_enabled	Whether to enable/disable auto-tune	bool	true	no
auto_tune_options	Configuration block for auto-tune options. The maintenance schedule block is required if rollback_on_disable is set to DEFAULT_ROLLBACK. The start_at field must be a time and date in RFC3339 format	object({ maintenance_schedule = optional(list(object({ start_at = string duration = object({ value = number }) cron_expression_for_recurrence = optional(string) }))) rollback_on_disable = string })	{ "maintenance_schedule": [], "rollback_on_disable": "NO_ROLLBACK" }	no
availability_zones	The number of availability zones for the OpenSearch cluster. Valid values: 1, 2 or 3.	number	3	no
cluster_domain	The hosted zone name of the OpenSearch cluster.	string	n/a	yes
cluster_domain_private	Indicates whether to create records in a private (true) or public (false) zone	bool	false	no
cluster_name	The name of the OpenSearch cluster.	string	"opensearch"	no
cluster_version	The version of OpenSearch to deploy.	string	"2.11"	no
cognito_options	Configuration block for authenticating Kibana with Cognito.	map(string)	{}	no
cognito_options_enabled	Whether Amazon Cognito authentication with Kibana is enabled or not.	bool	false	no
composable_index_template_files	A set of all composable index template files to create.	set(string)	[]	no
composable_index_templates	A map of all composable index templates to create.	map(any)	{}	no
create_service_role	Indicates whether to create the service-linked role. See https://docs.aws.amazon.com/opensearch-service/latest/developerguide/slr.html	bool	true	no
custom_endpoint	Fully qualified domain for your custom endpoint. If not specified, then it defaults to <cluster_name>.<cluster_domain>	string	null	no
custom_endpoint_certificate_arn	The ARN of the custom ACM certificate.	string	""	no
ebs_enabled	Indicates whether attach EBS volumes to the data nodes.	bool	false	no
ebs_iops	The baseline input/output (I/O) performance of EBS volumes attached to data nodes.	number	3000	no
ebs_throughput	The throughput (in MiB/s) of the EBS volumes attached to data nodes. Valid values are between 125 and 1000.	number	125	no
ebs_volume_size	The size of EBS volumes attached to data nodes (in GiB).	number	10	no
ebs_volume_type	The type of EBS volumes attached to data nodes.	string	"gp3"	no
encrypt_at_rest_enabled	Configuration block for encrypt at rest options	bool	true	no
encrypt_kms_key_id	The KMS key ID to encrypt the OpenSearch cluster with. If not specified, then it defaults to using the AWS OpenSearch Service KMS key.	string	""	no
hot_instance_count	The number of dedicated hot nodes in the cluster.	number	3	no
hot_instance_type	The type of EC2 instances to run for each hot node. A list of available instance types can you find at https://aws.amazon.com/en/opensearch-service/pricing/#On-Demand_instance_pricing	string	"r6gd.4xlarge.elasticsearch"	no
index_files	A set of all index files to create.	set(string)	[]	no
index_template_files	A set of all index template files to create.	set(string)	[]	no
index_templates	A map of all index templates to create.	map(any)	{}	no
indices	A map of all indices to create.	map(object({ number_of_shards = optional(number) number_of_replicas = optional(number) refresh_interval = optional(string) mappings = optional(any, {}) aliases = optional(any, {}) analysis_analyzer = optional(string) analysis_char_filter = optional(string) analysis_filter = optional(string) analysis_normalizer = optional(string) analysis_tokenizer = optional(string) analyze_max_token_count = optional(string) auto_expand_replicas = optional(string) blocks_metadata = optional(bool) blocks_read = optional(bool) blocks_read_only = optional(bool) blocks_read_only_allow_delete = optional(bool) blocks_write = optional(bool) codec = optional(string) default_pipeline = optional(string) gc_deletes = optional(string) highlight_max_analyzed_offset = optional(string) include_type_name = optional(string) index_similarity_default = optional(string) indexing_slowlog_level = optional(string) indexing_slowlog_source = optional(string) indexing_slowlog_threshold_index_debug = optional(string) indexing_slowlog_threshold_index_info = optional(string) indexing_slowlog_threshold_index_trace = optional(string) indexing_slowlog_threshold_index_warn = optional(string) load_fixed_bitset_filters_eagerly = optional(bool) max_docvalue_fields_search = optional(string) max_inner_result_window = optional(string) max_ngram_diff = optional(string) max_refresh_listeners = optional(string) max_regex_length = optional(string) max_rescore_window = optional(string) max_result_window = optional(string) max_script_fields = optional(string) max_shingle_diff = optional(string) max_terms_count = optional(string) number_of_routing_shards = optional(string) rollover_alias = optional(string) routing_allocation_enable = optional(string) routing_partition_size = optional(string) routing_rebalance_enable = optional(string) search_idle_after = optional(string) search_slowlog_level = optional(string) search_slowlog_threshold_fetch_info = optional(string) search_slowlog_threshold_fetch_debug = optional(string) search_slowlog_threshold_fetch_trace = optional(string) search_slowlog_threshold_fetch_warn = optional(string) search_slowlog_threshold_query_debug = optional(string) search_slowlog_threshold_query_info = optional(string) search_slowlog_threshold_query_trace = optional(string) search_slowlog_threshold_query_warn = optional(string) shard_check_on_startup = optional(string) sort_field = optional(string) sort_order = optional(string) }))	{}	no
ism_policies	A map of all ISM policies to create.	map(any)	{}	no
ism_policy_files	A set of all ISM policy files to create.	set(string)	[]	no
log_streams_enabled	Configuration for which log streams to enable sending logs to CloudWatch.	map(string)	{ "AUDIT_LOGS": "false", "ES_APPLICATION_LOGS": "false", "INDEX_SLOW_LOGS": "false", "SEARCH_SLOW_LOGS": "false" }	no
master_instance_count	The number of dedicated master nodes in the cluster.	number	3	no
master_instance_enabled	Indicates whether dedicated master nodes are enabled for the cluster.	bool	true	no
master_instance_type	The type of EC2 instances to run for each master node. A list of available instance types can you find at https://aws.amazon.com/en/opensearch-service/pricing/#On-Demand_instance_pricing	string	"r6gd.large.elasticsearch"	no
master_user_arn	The ARN for the master user of the cluster. If not specified, then it defaults to using the IAM user that is making the request.	string	""	no
node_to_node_encryption_enabled	Configuration block for node-to-node encryption options	bool	true	no
role_files	A set of all role files to create.	set(string)	[]	no
role_mapping_files	A set of all role mapping files to create.	set(string)	[]	no
role_mappings	A map of all role mappings to create.	map(any)	{}	no
roles	A map of all roles to create.	map(any)	{}	no
saml_enabled	Indicates whether to configure SAML for the OpenSearch dashboard.	bool	true	no
saml_entity_id	The unique Entity ID of the application in SAML Identity Provider.	string	""	no
saml_master_backend_role	This backend role receives full permissions to the cluster, equivalent to a new master role, but can only use those permissions within Dashboards.	string	null	no
saml_master_user_name	This username receives full permissions to the cluster, equivalent to a new master user, but can only use those permissions within Dashboards.	string	null	no
saml_metadata_content	The metadata of the SAML application in xml format.	string	""	no
saml_roles_key	Element of the SAML assertion to use for backend roles.	string	"http://schemas.microsoft.com/ws/2008/06/identity/claims/role"	no
saml_session_timeout	Duration of a session in minutes after a user logs in. Default is 60. Maximum value is 1,440.	number	60	no
saml_subject_key	Element of the SAML assertion to use for username.	string	"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name"	no
security_group_ids	The list of VPC security groups IDs to attach.	list(string)	[]	no
subnet_ids	The list of VPC subnet IDs to use.	list(string)	[]	no
tags	A map of tags to add to all resources.	map(string)	{}	no
vpc_enabled	Indicates whether the cluster is running inside a VPC.	bool	false	no
warm_instance_count	The number of dedicated warm nodes in the cluster.	number	3	no
warm_instance_enabled	Indicates whether ultrawarm nodes are enabled for the cluster.	bool	true	no
warm_instance_type	The type of EC2 instances to run for each warm node. A list of available instance types can you find at https://aws.amazon.com/en/elasticsearch-service/pricing/#UltraWarm_pricing	string	"ultrawarm1.large.elasticsearch"	no

Outputs

Name	Description
cluster_endpoint	The endpoint URL of the OpenSearch cluster.
cluster_name	The name of the OpenSearch cluster.
cluster_version	The version of the OpenSearch cluster.
kibana_endpoint	The endpoint URL of the OpenSearch dashboards.

License

Apache 2 Licensed. See LICENSE for full details.