Initial commit

.gitignore

@@ -0,0 +1,167 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# Juniper
+junos_sfnt.lic
+
+## VSCode
+.vscode
+*.code-workspace
+
+# Local History for Visual Studio Code
+.history/
+
+## MacOS
+# General
+.DS_Store
+.AppleDouble
+.LSOverride
+
+# Icon must end with two \r
+Icon
+
+# Thumbnails
+._*
+
+# Files that might appear in the root of a volume
+.DocumentRevisions-V100
+.fseventsd
+.Spotlight-V100
+.TemporaryItems
+.Trashes
+.VolumeIcon.icns
+.com.apple.timemachine.donotpresent
+
+# Directories potentially created on remote AFP share
+.AppleDB
+.AppleDesktop
+Network Trash Folder
+Temporary Items
+.apdisk

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Juniper Networks, Inc. 2020
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,187 @@
+# meshrr
+
+## Introduction
+*meshrr* is a demonstration-grade scale-out, hierarchically-capable, BGP route reflection methodology using Juniper cRPD and intended for deployment on Kubernetes.
+
+At this time and in the project's raw form, *meshrr* should not be considered for production environments. Sufficient testing, error handling, and routing daemon configuration best practices have not been implemented. Community contributions to improve these areas are appreciated.
+
+- [meshrr](#meshrr)
+  - [Introduction](#introduction)
+  - [Instructions](#instructions)
+    - [Prerequisites](#prerequisites)
+    - [Quickstart](#quickstart)
+    - [Environment Variables](#environment-variables)
+  - [Methodology](#methodology)
+- [Examples](#examples)
+  - [2 Regions with Hierarchical Route Reflectors](#2-regions-with-hierarchical-route-reflectors)
+    - [Description](#description)
+    - [Usage](#usage)
+
+## Instructions
+
+### Prerequisites
+1. An operational Kubernetes cluster with sufficient resources for the topology you wish to build.
+2. A *private* container registry accessible to your Kubernetes cluster.
+3. A cRPD license for the number of nodes you wish to deploy. At the time of writing, Juniper offers [free trial licenses](https://www.juniper.net/us/en/dm/crpd-trial/). Standard licenses are limited to 16 BGP peers and 4M RIB entries.
+
+### Quickstart
+1. (If required) modify [`juniper.conf.j2`](meshrr/juniper.conf.j2)
+2. Build and push your image. **Do not push to a public registry.**
+
+    ```bash
+    docker build -t <tag> meshrr
+    docker push <tag>
+    ```
+
+    e.g.
+    ```bash
+    docker build meshrr -t registry.example.com/meshrr/meshrr:latest
+    docker push registry.example.com/meshrr/meshrr:latest
+    ```
+3.  Either:
+    1. Pick an example topology from [`examples`](examples/) and modify the YAML files as required for your topology. Details for how to use examples and reasonable modifications are below in the [Examples](#Examples) section.
+    2. Create your own YAML files if you need a completely custom topology.
+4.  Populate the YAML files with the required information. You will need to, at a minimum, replace the following:
+    1. Names
+      1.  Service
+      2.  Labels (if following the 2regions-hrr example, your regions probably are not in Middle Earth)
+    2. [Environment Variables](#Environment-Variables)
+    3. Licensing mechanism. Examples here currently use a secret mounted as a volume mapped to `/config/license/safenet/junos_sfnt.lic`. This may be appropriate for bundle licenses.
+    4. Custom configuration Jinja2 templates loaded into ConfigMaps and mapped as volumes. See [Examples](#Examples).
+    5. Port mapping IP addresses (`hostIP`). No `hostIP` must be specified for instances only accessible within the cluster. Detailed strategy information to be defined in [Examples](#Examples).
+5.  Apply appropriate labels to the nodes:
+    ```bash
+    kubectl label nodes <node> <label1>=<value> <label2>=<value>
+    ```
+6.  Apply your configuration:
+    ```bash
+    kubectl [-n namespace] apply -f <file1>
+    kubectl [-n namespace] apply -f <file2>
+    ```
+
+### Environment Variables
+| Variable              | Required? | Description                                                                                                                                                                                                                                                                                                  |
+| --------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| POD_IP                | Yes       | The pod's IP address. Should be set by Kubernetes (`valueFrom: fieldRef: fieldPath: status.podIP`)                                                                                                                                                                                                           |
+| MESH_SERVICE_NAME     | Yes*      | The name of the mesh service. Set to the name of the headless Kubernetes service used for mesh BGP neighbor discovery. *Usually, a `MESH_SERVICE_NAME` is desirable. However, it may be skipped if there is an `UPSTREAM_SERVICE_NAME` in cases such as unmeshed regions learning routes from upstream HRRs. |
+| UPSTREAM_SERVICE_NAME | No        | The name of the upstream service. Set to the name of the headless Kubernetes service used for upstream BGP neighbor discovery. Defaults to `None`.                                                                                                                                                           |
+| KUBE_NAMESPACE        | No        | Optional name of the Kubernetes namespace. Defaults to `default`.                                                                                                                                                                                                                                            |
+| ENCRYPTED_ROOT_PW     | Yes       | Encrypted ($6) root password for cRPD                                                                                                                                                                                                                                                                        |
+| AUTONOMOUS_SYSTEM     | Yes       | ASN for the router.                                                                                                                                                                                                                                                                                          |
+| MESHRR_CLIENTRANGE    | Yes       | Range to allow. Currently, this accepts only one CIDR block. Format: `network/mask-length`                                                                                                                                                                                                                   |
+| SERVICE_ROOT_DOMAIN   | No        | Defaults to `svc.cluster.local`. You probably don't need to change this.                                                                                                                                                                                                                                     |
+
+## Methodology
+- Build container image based on crpd.
+    - Requires additional packages installed via `apt-get`:
+      - cron
+      - python3
+    - Builds crontab
+    - Sets up `runit-init.sh`
+- `runit-init.sh` initializes the environment:
+  - Saves environment variables, including the necessary pod's IP address, to `/etc/envvars`
+  - Sets up the cRPD configuration based on the template `juniper.conf.template`
+  - Calls `render_config.py` to create configuration file from Jinja2 template.
+- `update_peers.py` called every minute via cron.
+  - Uses a Kubernetes headless service DNS A records to detect peers.
+  New peers are added to config, removed peers are removed from config.
+  - Only occurs once a minute, so, given BGP timers, assume pod readiness 100 seconds from initiation.
+
+# Examples
+
+## 2 Regions with Hierarchical Route Reflectors
+![Example Topology](assets/2regions-hrr.png)
+![Example RR Hierarchy](assets/2regions-hrr-hierarchy.png)
+
+* Example YAML files: [examples/2regions-hrr](examples/2regions-hrr)
+
+### Description
+* This topology has two regions, `mirkwood` and `lothlorian`, and a `core` region.
+* Within a region, all cRRs (containerized route reflectors) are fully meshed to provide maximum visibility within the region.
+* All cRRs in a region other than `core` have BGP peerings with up to 2 `core` cRRs. (The limit of 2 is hard coded on upstream peer groups.)
+* **Redundancy groups and anycast addressing:**
+  * Each node is assigned to redundancy group `a` or `b`.
+  * For each region with neighbors outside the cluster, separate DaemonSets are created for `a` and `b`, each with a unique IP address for that [meshrr_region:redundancy_group] combination. This IP address is used for iBGP peering with neighbors outside the cluster.
+  * Each /32 is assigned to the loopback interface of *every* node in the [meshrr_region:redundancy_group] combination, and the routers connecting to the node have a static route to the /32 that is redistributed into the IGP. For example, in the above topology:
+    
+    | Node Region | MESHRR_REGION(s)           | Redundancy Group | Loopback Address(es)   |
+    | ----------- | -------------------------- | ---------------- | ---------------------- |
+    | Mirkwood    | mirkwood                   | a                | 172.19.1.1             |
+    | Mirkwood    | mirkwood                   | b                | 172.19.1.2             |
+    | Core        | core, mirkwood, lothlorien | a                | 172.19.1.1, 172.19.2.1 |
+    | Core        | core, mirkwood, lothlorien | b                | 172.19.1.2, 172.19.2.2 |
+    | Lothlorien  | lothlorien                 | a                | 172.19.2.1             |
+    | Lothlorien  | lothlorien                 | b                | 172.19.2.2             |
+
+### Usage
+1.  Follow the instructions in [Quickstart](#Quickstart) using the example YAML files in [examples/2regions-hrr]().
+
+2.  Create the necessary loopback IPs on each of the nodes based on redundancy group and region. Internal-only MESHRR_REGIONs do not require a configured loopback IP.
+
+    ```bash
+    sudo ip address add 172.19.1.1 dev lo
+    ```
+
+3.  Configure your routers servicing the nodes with static routes redistributed into your IGP for the node loopback addresses.
+Configuration on the router servicing a Mirkwood A node may look like:
+
+    ##### Junos
+    ```junos
+    routing-options {
+        static {
+            route 172.0.1.1/32 next-hop <Node IP>;
+        }
+    }
+    policy-options {
+        policy-statement RRSTATIC-TO-ISIS {
+            from {
+                protocol static;
+                route-filter 172.19.1.1/32 exact;
+            }
+            then accept;
+        }
+    }
+    protocols {                              
+        isis {
+            export RRSTATIC-TO-ISIS;
+        }
+    }
+    ```
+
+    ##### IOS-XR
+    ```iox-xr
+    router static
+      address-family ipv4 unicast
+      172.0.1.1/32 <Node IP>
+    !
+    route-policy STATIC-TO-ISIS
+      if destination in (172.19.1.1/32) then
+        pass
+      endif
+    end-policy
+    !
+    router isis ISIS
+      redistribute static level-2 route-policy STATIC-TO-ISIS
+    !
+    ```
+
+4.  Modify configuration templates as necessary. [`meshrr/juniper.conf.j2`](meshrr/juniper.conf.j2) will be loaded to all instances by default, but customizations on a per-deployment/per-daemonset basis should be performed on other J2 files (see [`mirkwood-config.j2`](examples/2regions-hrr/templates/mirkwood-config.j2) and [`lothlorien-config.j2`](examples/2regions-hrr/templates/lothlorien-config.j2).
+Apply these configuration templates as ConfigSets for any cases that require customization as so:
+
+    ```bash
+    k create configmap mirkwood-config \
+      --from-file=config=examples/2regions-hrr/templates/mirkwood-config.j2 \
+      -o yaml --dry-run=client |
+      k apply -f -
+    ```
+
+    ```bash
+    k create configmap lothlorien-config \
+      --from-file=config=examples/2regions-hrr/templates/lothlorien-config.j2 \
+      -o yaml --dry-run=client |
+      k apply -f -
+    ```
+
+    These ConfigMaps are mounted as volumes in the corresponding DaemonSets.
+
+5.  Modify the YAML files to your needs. At the least, `<registryURL>` will need to be replaced to reference your private registry. Load the YAML files for the DaemonSets and Services into Kubernetes as per [Quickstart](#Quickstart).