Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add Layer 2 example including interconnection and VRF #232

Merged
merged 16 commits into from
Oct 29, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions examples/layer2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Layer 2 networking with Equinix Metal

This example demonstrates the use of the `equinix.cloud.metal_connection`, `equinix.cloud.metal_device`, and `equinix.cloud.metal_port` modules--as well as a variety of AWS modules--to configure Layer 2 connectivity from an Equinix Metal device to AWS S3 over a Metal-billed Fabric interconnection.

## Overview

The [pre-Fabric playbook](pre_fabric.yml) creates a new project, a VLAN, a VRF, a VRF Metal Gateway, and a device, converts the device to [hybrid bonded mode](https://deploy.equinix.com/developers/docs/metal/layer2-networking/overview/#network-configuration-types), and then creates a Metal-billed VRF interconnection and configures BGP peering settings on the interconnection's virtual circuit.

Manual intervention is needed in order to finish setting up the interconnection and accept the Direct Connect request in AWS.

The [post-Fabric playbook](post_fabric.yml) creates a new VPC, a VPC endpoint for S3, and a Virtual Private Gateway attached to the specified Direct Connect, and configures BGP peering between the Direct Connect and your Metal VRF.

## Prerequisites

Before running the playbook, you will need to have the following:

- [Ansible installed on your local machine.](https://docs.ansible.com/ansible/latest/installation_guide/installation_distros.html)
- The Community.Aws, and Equinix Ansible Collections installed. You can install them using the following commands:
```bash
ansible-galaxy collection install equinix.cloud
ctreatma marked this conversation as resolved.
Show resolved Hide resolved
ansible-galaxy collection install community.aws
```
- You will also need to ensure that the necessary Python libraries are installed:
```bash
# Install Equinix Ansible collection dependencies
pip install -r https://raw.githubusercontent.com/equinix/ansible-collection-equinix/v0.11.1/requirements.txt
# Install AWS collection and Ansible IP function dependencies
pip install boto3 netaddr
```
- An [Equinix Metal API token](https://deploy.equinix.com/developers/docs/metal/identity-access-management/api-keys/). You can obtain an API token from the Equinix Metal Portal. Set the environment variable METAL_AUTH_TOKEN to your API token:
```bash
export METAL_AUTH_TOKEN=your_api_token_here
```

## Variables

You can customize some variables, such as Equinix Metal device hostname and IP ranges for Equinix Metal and AWS, from [vars/vars.yml](vars/vars.yml).

## Running the Playbooks

This example contains multiple playbooks and requires manual intervention between the playbooks.

To create the Equinix Metal infrastructure for this example, navigate to the directory containing the playbook file `pre_fabric.yml` and run the following command:

```bash
ansible-playbook pre_fabric.yml -extra-vars "bgp_md5_password=<some_value>"
```

*NOTE:* The API performs some validation on the md5 for BGP. For the latest rules refer to [the VRF virtual circuit API docs](https://deploy.equinix.com/developers/api/metal/#tag/Interconnections/operation/updateVirtualCircuit). As of this writing, the md5:
* must be 10-20 characters long
* may not include punctuation
* must be a combination of numbers and letters
* must contain at least one lowercase, uppercase, and digit character

The last task in the `pre_fabric.yml` playbook will print out the service token for your Metal connection:

```bash
TASK [print service token to redeem in Fabric portal] **************************************************************************
ok: [localhost] => {
"connection.service_tokens[0].id": "<service_token_id>"
}
```

After the Equinix Metal infrastructure is created, you will need to redeem the service token for your connection in the [Equinix portal](https://portal.equinix.com). Navigate to Fabric -> Connect to Provider, choose AWS, and finally AWS Direct Connect. Choose Primary, put in your account number, choose the metro, click next, then choose "Service Token" from the drop down, and put in the service token. You will be prompted to name your connection; **take note of the name you use**, you will need it for the next playbook.

ctreatma marked this conversation as resolved.
Show resolved Hide resolved
To finish setting up the AWS infrastructure, run the following command which will accept the direct connect request in AWS; wait for the connection to become active; create an AWS VPC, VPC endpoint, and VPN gateway; create a virtual interface connecting the VPC to your direct connect, and configure the Metal side of your interconnection to connect to the virtual interface in AWS:

```bash
ansible-playbook post_fabric.yml --extra-vars "bgp_md5_password=<some_value>" --extra-vars "aws_connection_name=<your_direct_connect_name>"
cprivitere marked this conversation as resolved.
Show resolved Hide resolved
```

The last task in the `post_fabric.yml` playbook will print the DNS hostname for your S3 VPC endpoint:

```bash
TASK [print DNS name for VPC endpoint] *****************************************************************************************
ok: [localhost] => {
"msg": "vpce-<some_id>.s3.us-west-1.vpce.amazonaws.com"
}
```



## Testing the VPC endpoint and interconnection

The DNS entry for your VPC endpoint is public, so you can look up the corresponding IP address from any Internet-connected computer. You will see that it resolves to an IP address within your private VPC address space (the example below uses the default VPC CIDR for this module, `172.16.0.0/16`):

```bash
$ dig vpce-<some_id>.s3.us-west-1.vpce.amazonaws.com
# ...
;; ANSWER SECTION:
vpce-<some_id>.s3.us-west-1.vpce.amazonaws.com. 60 IN A 172.16.94.176
# ...
```
ctreatma marked this conversation as resolved.
Show resolved Hide resolved

Since this address resolves to an IP within your VPC, though, you can only connect to it from an EC2 instance in your VPC or from the Metal device you deployed earlier.

SSH in to the Metal device that was created by the `pre_fabric.yml` playbook.

Install the AWS CLI:

```bash
$ apt install -y awscli
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be done via userdata but I think it's a better flow to save the entire AWS CLI setup until the end.

```

Configure your [AWS CLI credentials](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html). For example, if you want to store your credentials in environment variables, it will look something like this:

```bash
$ export AWS_ACCESS_KEY_ID=<some_aws_key_id>
$ export AWS_SECRET_ACCESS_KEY=<some_aws_access_key>
$ export AWS_DEFAULT_REGION=us-west-1
```

You can now use the AWS CLI with your VPC endpoint to interact with the S3 service by adding the `bucket.` prefix to your VPC endpoint hostname:

```bash
$ aws s3 ls --endpoint-url https://bucket.vpce-<some_id>.s3.us-west-1.vpce.amazonaws.com
2021-03-22 11:13:54 <some_bucket>
2021-03-22 11:13:54 <some_other_bucket>
...
```

You can learn about other usages of the S3 VPC endpoint with AWS CLI in [the AWS PrivateLink docs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html).
146 changes: 146 additions & 0 deletions examples/layer2/post_fabric.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
---
# NOTE: this playbook should be run _after_:
# 1. Running the pre_fabric.yml playbook
# 2. Redeeming the Fabric service token in the Fabric portal
- name: Equinix Layer 2 example -- AWS resources
hosts: localhost
gather_facts: no
tasks:
- name: Include the required variables
include_vars: "vars/vars.yml"

- name: confirm the direct connect
community.aws.directconnect_confirm_connection:
region: "{{ aws_region }}"
name: "{{ aws_connection_name }}"
register: confirm_response

- name: wait for the direct connect to become available
community.aws.directconnect_connection:
region: "{{ aws_region }}"
name: "{{ aws_connection_name }}"
# The below attributes are required by the community.aws.directconnect_connection
# module but do not appear to be used, since this works for looking up the connection
location: "dummy"
bandwidth: "1Gbps"
state: present
register: connection_response
until: connection_response.connection.connection_state == "available"
retries: 10
delay: 60

- name: create a VPC
amazon.aws.ec2_vpc_net:
region: "{{ aws_region }}"
name: "{{ vpc_name }}"
cidr_block: "{{ aws_network_cidr }}"
register: created_vpc

- name: Create a subnet in the VPC
amazon.aws.ec2_vpc_subnet:
region: "{{ aws_region }}"
vpc_id: "{{ created_vpc.vpc.id }}"
cidr: "{{ aws_network_cidr }}"
register: created_subnet

- name: create security group for VPC endpoint
amazon.aws.ec2_security_group:
region: "{{ aws_region }}"
vpc_id: "{{ created_vpc.vpc.id }}"
name: "{{ security_group_name }}"
description: sg for Equinix Ansible Layer 2 example
rules:
- proto: all
cidr_ip: 0.0.0.0/0
rule_desc: allow all traffic
register: created_sg

- name: Create new VPC endpoint for S3
amazon.aws.ec2_vpc_endpoint:
region: "{{ aws_region }}"
vpc_id: "{{ created_vpc.vpc.id }}"
service: "com.amazonaws.{{ aws_region }}.s3"
vpc_endpoint_type: "Interface"
vpc_endpoint_subnets:
- "{{ created_subnet.subnet.id }}"
vpc_endpoint_security_groups:
- "{{ created_sg.group_id }}"
register: created_vpc_endpoint

- name: Create a new VGW attached to the VPC
community.aws.ec2_vpc_vgw:
region: "{{ aws_region }}"
vpc_id: "{{ created_vpc.vpc.id }}"
name: "{{ vpc_gateway_name }}"
register: created_vgw

- name: Create an association between VGW and connection
community.aws.directconnect_virtual_interface:
region: "{{ aws_region }}"
state: present
name: "{{ directconnect_vif_name }}"
public: false
connection_id: "{{ connection_response.connection.connection_id }}"
vlan: "{{ connection_response.connection.vlan }}"
virtual_gateway_id: "{{ created_vgw.vgw.id }}"
customer_address: "{{ metal_peering_ip }}/30"
amazon_address: "{{ aws_peering_ip }}/30"
bgp_asn: "{{ metal_side_asn }}"
authentication_key: "{{ bgp_md5_password }}"
register: created_vif
until: created_vif.virtual_interface_state == "available"
retries: 10
delay: 60

- name: Enable VGW route propagation
amazon.aws.ec2_vpc_route_table:
region: "{{ aws_region }}"
vpc_id: "{{ created_vpc.vpc.id }}"
propagating_vgw_ids:
- "{{ created_vgw.vgw.id }}"

- name: Look up the project we created earlier
equinix.cloud.metal_project:
name: "{{ project_name }}"
register: project

- name: Look up the VRF we created earlier
equinix.cloud.metal_vrf:
name: "{{ vrf_name }}"
metro: "{{ metro }}"
local_asn: "{{ metal_side_asn }}"
ip_ranges:
- "{{ vrf_peering_ip_range }}"
- "{{ vrf_gateway_ip_range }}"
project_id: "{{ project.id }}"
register: vrf

- name: look up the Metal-billed VRF interconnection we created earlier
equinix.cloud.metal_connection:
project_id: "{{ project.id }}"
metro: "{{ metro }}"
name: "{{ interconnection_name }}"
type: "shared"
speed: "50Mbps"
service_token_type: a_side
redundancy: primary
vrfs:
- "{{ vrf.id }}"
register: connection

- name: Configure BGP for interconnection virtual circuit
equinix.cloud.metal_virtual_circuit:
id: "{{ connection.ports[0].virtual_circuits[0].id }}"
peer_asn: "{{ created_vif.amazon_side_asn }}"
customer_ip: "{{ aws_peering_ip }}"
metal_ip: "{{ metal_peering_ip }}"
subnet: "{{ vrf_vc_peering_ip_range }}"
md5: "{{ bgp_md5_password }}"
# The metal_virtual_circuit module requires this parameter
# in order to know that the circuit is a VRF circuit and
# not a VLAN circuit
vrf: "{{ vrf.id }}"
cprivitere marked this conversation as resolved.
Show resolved Hide resolved

- name: print DNS name for VPC endpoint
debug:
msg: "{{ created_vpc_endpoint.result.dns_entries[0].dns_name | replace('*.', '') }}"
102 changes: 102 additions & 0 deletions examples/layer2/pre_fabric.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
- name: Equinix Metal Example Playbook
hosts: localhost
gather_facts: no
tasks:
- name: Include the required variables
include_vars: "vars/vars.yml"

# Equinix resources
- name: Create a project
equinix.cloud.metal_project:
name: "{{ project_name }}"
register: project

- name: create a vlan
equinix.cloud.metal_vlan:
project_id: "{{ project.id }}"
metro: "{{ metro }}"
vxlan: "1234"
register: vlan

- name: create a VRF
equinix.cloud.metal_vrf:
name: "{{ vrf_name }}"
metro: "{{ metro }}"
local_asn: "{{ metal_side_asn }}"
ip_ranges:
- "{{ vrf_peering_ip_range }}"
- "{{ vrf_gateway_ip_range }}"
project_id: "{{ project.id }}"
register: vrf

- name: create a VRF IP reservation
equinix.cloud.metal_reserved_ip_block:
project_id: "{{ project.id }}"
vrf_id: "{{ vrf.id }}"
type: "vrf"
metro: "{{ metro }}"
network: "{{ vrf_gateway_ip_range | split('/') | first }}"
cidr: "{{ vrf_gateway_ip_range | split('/') | last }}"
register: vrf_ip_reservation

- name: create a VRF Metal Gateway
equinix.cloud.metal_gateway:
project_id: "{{ project.id }}"
ip_reservation_id: "{{ vrf_ip_reservation.id }}"
virtual_network_id: "{{ vlan.id }}"
register: gateway

# Create a device
- name: Create a device
equinix.cloud.metal_device:
project_id: "{{ project.id }}"
metro: "{{ metro }}"
hostname: "{{ device_hostname }}"
operating_system: "{{ operating_system }}"
plan: "{{ plan }}"
state: present
userdata: |2
#!/bin/bash

cat <<EOF >> /etc/network/interfaces
auto bond0.{{vlan.vxlan}}
iface bond0.{{vlan.vxlan}} inet static
address {{ vrf_gateway_ip_range | ansible.utils.nthhost(2) }}
netmask {{ vrf_ip_reservation.netmask }}
post-up route add -net {{ vrf_gateway_ip_range }} gw {{ vrf_ip_reservation.gateway }}
post-up route add -net {{ aws_network_cidr }} gw {{ vrf_ip_reservation.gateway }}
EOF

systemctl restart networking
register: device

- name: capture port ids for device
set_fact:
bond_port_id: "{{ device.network_ports | selectattr('name', 'match', 'bond0') | map(attribute='id') | first }}"
eth1_port_id: "{{ device.network_ports | selectattr('name', 'match', 'eth1') | map(attribute='id') | first }}"

- name: convert bond port to hybrid bonded mode
equinix.cloud.metal_port:
id: "{{ bond_port_id }}"
bonded: true
layer2: false
vlan_ids:
- "{{ vlan.id }}"

- name: create a Metal-billed VRF interconnection
equinix.cloud.metal_connection:
project_id: "{{ project.id }}"
metro: "{{ metro }}"
name: "{{ interconnection_name }}"
type: "shared"
speed: "50Mbps"
service_token_type: a_side
redundancy: primary
vrfs:
- "{{ vrf.id }}"
register: connection

- name: print service token to redeem in Fabric portal
debug:
var: connection.service_tokens[0].id
23 changes: 23 additions & 0 deletions examples/layer2/vars/vars.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Equinix variables
project_name: my_metal_layer2_project
device_hostname: layer2-device
metro: sv
operating_system: ubuntu_20_04
plan: c3.small.x86
interconnection_name: "layer2 interconnection"
vrf_name: "example-vrf"
vrf_gateway_ip_range: 192.168.200.0/25
vrf_peering_ip_network: 169.254.0.0
vrf_peering_ip_range: "{{ vrf_peering_ip_network }}/29"
vrf_vc_peering_ip_range: "{{ vrf_peering_ip_network }}/30"
metal_peering_ip: "{{ vrf_vc_peering_ip_range | ansible.utils.nthhost(2) }}"
aws_peering_ip: "{{ vrf_vc_peering_ip_range | ansible.utils.nthhost(1) }}"
metal_side_asn: 65100
fabric_connection_name: ansible-layer2-example
# AWS variables
aws_network_cidr: 172.16.0.0/16
ctreatma marked this conversation as resolved.
Show resolved Hide resolved
aws_region: us-west-1
directconnect_vif_name: ansible-equinix-layer2-example
security_group_name: ansible-equinix-layer2-example
vpc_gateway_name: ansible-equinix-layer2-example
vpc_name: ansible-equinix-layer2-example