Skip to content

Commit

Permalink
working Open vSwitch autostart (kicked IPSec for the moment, didn't w…
Browse files Browse the repository at this point in the history
…ork with it)
  • Loading branch information
michelkaeser committed Nov 6, 2015
1 parent d305840 commit 0d07a5e
Showing 1 changed file with 43 additions and 27 deletions.
70 changes: 43 additions & 27 deletions docs/install/networking/openvswitch.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,63 +12,79 @@ Open vSwitch packages are available in all major Linux distributions, so the ins
On Debian based distributions you'll have to run:

```bash
apt-get install -y openvswitch-switch openvswitch-ipsec
apt-get install -y openvswitch-switch
```

> If the nodes are protected by a firewall, make sure to open the ports `500, 1723 and 4500` as well as to allow the `esp` and `ah` IP protocols.
## Setting up the interface

Now that Open vSwitch is installed, we need to create an Open vSwitch bridge interface, which will act as the nodes' virtual switch. All of the following commands need to be executed on every node (if not stated otherwise).



To create the interface, issue:
To create the interface and make it auto-start on boot, issue:

```bash
$ ovs-vsctl add-br ovsbr0
$ ovs-vsctl set bridge ovsbr0 stp_enable=true
$ nano /etc/network/interfaces.d/coco-project
```

To assign an IPv4 address from the picked range to the created `ovsbr0` interface, execute the following statements:
and append the following lines:

```bash
$ ifconfig ovsbr0 up 192.168.0.1 netmask 255.255.255.0
$ ifconfig ovsbr0 mtu 1420
auto coco_br0
allow-ovs coco_br0
iface coco_br0 inet static
address 192.168.0.1
netmask 255.255.255.0
mtu 1420
ovs_type OVSBridge
ovs_extra set bridge ${IFACE} stp_enable=true
```

> `192.168.0.1` is the internal only IPv4 address of the current node. Make sure every node has another IP address. Usually the master node will have `x.x.x.1`.
> ––––
> `255.255.255.0` is the netmask of the private network. If you plan to deploy more than 254 nodes, pick a `/16` or `/8` range.
> ––––
> These commands are best placed in `/etc/rc.local` so they are executed on boot. Make sure to put them before `exit 0`.
> `255.255.255.0` is the netmask of the private network. If you plan to deploy more than 254 nodes, pick a `/16` or `/8` range.
Complete the setup by adding the bridge to the internal Open vSwitch database too:

```bash
ovs-vsctl add-br coco_br0
```

## Establishing connections between the nodes

Open vSwitch is installed and running, but no connections between the nodes have been added yet. Don't worry, adding them is as simple as the installation was.

Basically, the following command needs to be executed on the two nodes between which the connection should be established. Executing that command instructs Open vSwitch to create and establish a `GRE over IPSec` connection beween the two nodes:
Basically, the following commands needs to be executed on the two nodes between which the connection should be established:

```bash
$ ovs-vsctl add-port ovsbr0 gre_master_slave1 -- set interface gre_master_slave1 type=ipsec_gre options:remote_ip=10.0.0.2 options:psk=coco
nano /etc/network/interfaces.d/coco-project
```

> `gre_master_slave1` is the connection's name. It must be unique and the same on both nodes.
> ––––
> `10.0.0.2` is the IPv4 address under which the remote node can be reached.
> ––––
> `psk=coco` is the password used to encrypt the connection.
and add an internal Open vSwitch port:

```bash
auto coco_gre1
allow-coco_br0 coco_gre1
iface coco_gre1 inet manual
ovs_bridge coco_br0
ovs_type OVSPort
ovs_extra set interface ${IFACE} type=gre options:remote_ip=10.0.0.2
```

For a minimal setup, you have to establish one connection to the master node at least. A full-meshed network might however perform better, so you're encouraged to establish additional connections between other nodes as well.
> `coco_gre1` is the connection's name. It must be unique and the same on both nodes.
> ––––
> `10.0.0.2` is the IPv4 address under which the remote node can be reached.
## Troubleshooting
Additionally, add the following line (or only the port if already there) to the bridge you created during the setup phase:

### 1. Connections are not established after a reboot
```bash
...
ovs_extra set bridge ${IFACE} stp_enable=true
ovs_ports coco_gre1 # newly added
```

We saw this quite often. The solution is to restart the Open vSwitch services on the nodes:
Last but not least, add the port to the database:

```bash
$ service openvswitch-ipsec restart && service openvswitch-switch restart
ovs-vsctl add-port coco_br0 coco_gre1
```

> Other services connecting to remote nodes via the internal network might need a restart as well, as soon as the connections have been established.
> To make sure the GRE connections are established before running i.e. custom scripts, you can place `ping -c 1 10.0.0.2` in `/etc/rc.local` where `10.0.0.2` is the IPv4 address of the remote node you want to reach. Every command placed after this line will be able to communicate with the remote node.

0 comments on commit 0d07a5e

Please sign in to comment.