Skip to content

Commit

Permalink
feat: configure networking customization (#55)
Browse files Browse the repository at this point in the history
  • Loading branch information
@bissquit
bissquit authored Aug 28, 2023
1 parent 6409f2d commit 97a725c
Show file tree
Hide file tree
Showing 12 changed files with 78 additions and 56 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
- name: List molecule scenarios
id: set-matrix
run: |
MOLECULE_SCENARIOS=$(find molecule/ -mindepth 1 -type d -exec basename {} \; | jq -ncR '[inputs]')
MOLECULE_SCENARIOS=$(find molecule/ -maxdepth 1 -mindepth 1 -type d -exec basename {} \; | jq -ncR '[inputs]')
echo "molecule-scenarios=${MOLECULE_SCENARIOS}" >> $GITHUB_OUTPUT
tests:
Expand Down
3 changes: 2 additions & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,5 @@ test:
molecule test --all

lint:
molecule lint
yamllint .
ansible-lint
49 changes: 27 additions & 22 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,40 +7,45 @@ Role initializes Kubernetes cluster from the scratch, adds all defined control-p
Requirements
------------

- [container runtime](https://kubernetes.io/docs/setup/production-environment/container-runtimes/) should be installed (containerd is now the only supported container runtime)
- `iproute2` package to collect network facts for Debian-like OS
- provide all required certificates and appropriate etcd endpoint if you use external etcd cluster
- [container runtime](https://kubernetes.io/docs/setup/production-environment/container-runtimes/). Containerd is now the only supported container runtime
- [CNI plugin](https://github.com/containernetworking/cni)
- `iproute2` package (collect network facts for Debian-like OS)
- Client certificates (if you use external etcd cluster with TLS)

Role Variables
--------------

All variables are defined as defaults in [defaults/main.yml](defaults/main.yml) and may be overrided.

| Name | Default value | Description |
|---------------------------------------|----------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------|
| `k8s_cluster_kubelet_version` | 1.26.0-00 | kubelet version |
| `k8s_cluster_kubeadm_version` | 1.26.0-00 | kubeadm version |
| `k8s_cluster_kubectl_version` | 1.26.0-00 | kubectl version |
| `k8s_cluster_apt_key_url` | https://packages.cloud.google.com/apt/doc/apt-key.gpg | |
| `k8s_cluster_apt_repository` | deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main | |
| `k8s_cluster_kubernetes_version` | 1.26.0 | Kubernetes version |
| `k8s_cluster_kubelet_config_root_dir` | /etc/kubernetes | Default kubelet configuration directory |
| `k8s_cluster_node_type` | worker | Default node type. If you need to init or join master, you should set this variable to 'master' |
| `k8s_cluster_node_name` | inventory_hostname | Name of k8s node |
| `k8s_cluster_initial_master` | false | This variable identifies initial master node to initialize cluster. It should be assigned to the only node with 'true' value |
| `k8s_cluster_init_configuration` | See [defaults/main.yml](defaults/main.yml) | Represents `kind: InitConfiguration` of the cluster in pure yaml format |
| `k8s_cluster_cluster_configuration` | See [defaults/main.yml](defaults/main.yml) | Represents `kind: ClusterConfiguration` of the cluster in pure yaml format |
| `k8s_cluster_kubelet_configuration` | See [defaults/main.yml](defaults/main.yml) | Represents `kind: KubeletConfiguration` in pure yaml format |
| `k8s_cluster_kubeproxy_configuration` | "" | Represents `kind: KubeproxyConfiguration` in pure yaml format |
| `k8s_cluster_join_configuration` | See [defaults/main.yml](defaults/main.yml) | Represents `kind: JoinConfiguration` of the cluster in pure yaml format |
| `k8s_cluster_flannel_apply` | https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml | Flannel configuration. Will be removed or significantly changed in the future |
| Name | Default value | Description |
|--------------------------------------------|----------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `k8s_cluster_kubelet_version` | 1.26.0-00 | kubelet version |
| `k8s_cluster_kubeadm_version` | 1.26.0-00 | kubeadm version |
| `k8s_cluster_kubectl_version` | 1.26.0-00 | kubectl version |
| `k8s_cluster_apt_key_url` | https://packages.cloud.google.com/apt/doc/apt-key.gpg | Apt key url |
| `k8s_cluster_apt_repository` | deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main | Apt repository |
| `k8s_cluster_kubernetes_version` | 1.26.0 | Kubernetes version |
| `k8s_cluster_kubelet_config_root_dir` | /etc/kubernetes | Default kubelet configuration directory |
| `k8s_cluster_node_type` | worker | Default node type. If you need to init or join master, you should set this variable to 'master' |
| `k8s_cluster_node_name` | inventory_hostname | Name of k8s node |
| `k8s_cluster_initial_master` | false | This variable identifies initial master node to initialize cluster. It should be assigned to the only node with 'true' value |
| `k8s_cluster_init_configuration` | See [defaults/main.yml](defaults/main.yml) | Represents `kind: InitConfiguration` of the cluster in pure yaml format |
| `k8s_cluster_cluster_configuration` | See [defaults/main.yml](defaults/main.yml) | Represents `kind: ClusterConfiguration` of the cluster in pure yaml format |
| `k8s_cluster_kubelet_configuration` | See [defaults/main.yml](defaults/main.yml) | Represents `kind: KubeletConfiguration` in pure yaml format |
| `k8s_cluster_kubeproxy_configuration` | "" | Represents `kind: KubeproxyConfiguration` in pure yaml format |
| `k8s_cluster_join_configuration` | See [defaults/main.yml](defaults/main.yml) | Represents `kind: JoinConfiguration` of the cluster in pure yaml format |
| `k8s_cluster_flannel_apply` | https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml | Default Flannel manifest. Read more about [Flannel](https://github.com/flannel-io/flannel) |
| `k8s_cluster_custom_networking_tasks_path` | "" | Path to file with a set of tasks to configure networking. It could be a set of tasks or even `include_role` statement. It will run only once during cluster initialization |

\* except `apiVersion` and `kind` fields

Dependencies
------------

Use any role to install containerd
We recommend the following roles to install Containerd and CNI plugin:

- geerlingguy.containerd
- githubixx.cni

Example Playbook
----------------
Expand Down
3 changes: 2 additions & 1 deletion defaults/main.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ k8s_cluster_kubernetes_version: 1.26.0
k8s_cluster_kubelet_config_root_dir: '/etc/kubernetes'

k8s_cluster_node_type: worker
k8s_cluster_initial_master: false
k8s_cluster_node_name: "{{ inventory_hostname }}"
k8s_cluster_initial_master: false

# https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-InitConfiguration
k8s_cluster_init_configuration:
Expand Down Expand Up @@ -88,3 +88,4 @@ k8s_cluster_join_configuration:

# https://github.com/flannel-io/flannel
k8s_cluster_flannel_apply: https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
k8s_cluster_custom_networking_tasks_path: ""
3 changes: 0 additions & 3 deletions molecule/default/molecule.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,6 @@ provisioner:
k8s_cluster_initial_master: true
verifier:
name: ansible
lint: |
yamllint .
ansible-lint
scenario:
test_sequence:
- dependency
Expand Down
3 changes: 0 additions & 3 deletions molecule/default_ext-etcd-sec/molecule.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -126,9 +126,6 @@ provisioner:
- "{{ etcd_cert_dir }}/peer.pem"
verifier:
name: ansible
lint: |
yamllint .
ansible-lint
scenario:
test_sequence:
- dependency
Expand Down
3 changes: 3 additions & 0 deletions molecule/end-to-end/molecule.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,9 @@ provisioner:
dnsDomain: cluster.local
kubernetesVersion: "{{ k8s_cluster_kubernetes_version }}"
controlPlaneEndpoint: "{{ k8s_cluster_control_plane_endpoint }}:6443"
# custom networking
k8s_cluster_flannel_apply: ""
k8s_cluster_custom_networking_tasks_path: "network/custom-networking.yml"
lb-etcd:
haproxy_frontend_name: 'etcd'
haproxy_frontend_bind_address: '*'
Expand Down
9 changes: 9 additions & 0 deletions molecule/end-to-end/network/custom-networking.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---

- name: Configure Calico networking
ansible.builtin.shell: |
kubectl apply \
-f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml \
--kubeconfig=/etc/kubernetes/admin.conf \
> /tmp/calico.log 2>&1
changed_when: False
8 changes: 4 additions & 4 deletions requirements.txt
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
ansible
ansible==5.10.0
ansible-core==2.12.10
ansible-compat==3.0.2
molecule==4.0.4
molecule[docker]
molecule[lint]
ansible-lint
yamllint
ansible-lint==6.16.2
yamllint
3 changes: 0 additions & 3 deletions requirements.yml
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
---

roles:
- name: geerlingguy.docker
version: 6.1.0

- name: geerlingguy.containerd
version: 1.3.1

Expand Down
27 changes: 27 additions & 0 deletions tasks/configure-networking.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---

# https://github.com/flannel-io/flannel
- name: Configure default (Flannel) networking
ansible.builtin.shell: |
kubectl apply \
-f {{ k8s_cluster_flannel_apply }} \
--kubeconfig=/etc/kubernetes/admin.conf \
> /tmp/flannel.log 2>&1
changed_when: False
when:
- k8s_cluster_custom_networking_tasks_path == ""
- k8s_cluster_flannel_apply != ""

- name: Configure custom networking
ansible.builtin.include_tasks: "{{ k8s_cluster_custom_networking_tasks_path }}"
when: k8s_cluster_custom_networking_tasks_path != ""

- name: Check master node until ready
ansible.builtin.shell: |
kubectl get node {{ k8s_cluster_node_name }} \
--kubeconfig=/etc/kubernetes/admin.conf -o json | jq -Mr '.status.conditions[4].status'
register: kubectl_response
changed_when: false
until: kubectl_response.stdout == "True"
retries: 48
delay: 5
21 changes: 3 additions & 18 deletions tasks/init-control-plane.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,21 +49,6 @@
- name: Stop play for all hosts because cluster is not initialized
ansible.builtin.meta: end_play

# https://github.com/flannel-io/flannel
- name: Configure Flannel networking
ansible.builtin.shell: |
kubectl apply \
-f {{ k8s_cluster_flannel_apply }} \
--kubeconfig=/etc/kubernetes/admin.conf \
> /tmp/flannel.log 2>&1
changed_when: False

- name: Check master node until ready
ansible.builtin.shell: |
kubectl get node {{ k8s_cluster_node_name }} \
--kubeconfig=/etc/kubernetes/admin.conf -o json | jq -Mr '.status.conditions[4].status'
register: kubectl_response
changed_when: false
until: kubectl_response.stdout == "True"
retries: 48
delay: 5
- name: Configure cluster networking
ansible.builtin.include_tasks: configure-networking.yml
when: k8s_cluster_initial_master | bool

0 comments on commit 97a725c

Please sign in to comment.