docs: initial commit

This commit adds initial documentation on how to run development version of CAPT and how to create a workload cluster using it. Closes tinkerbell#13 Signed-off-by: Mateusz Gozdek <[email protected]>
parauliya · Jan 15, 2021 · 73ddefa · 73ddefa
1 parent 2e8a69f
commit 73ddefa
Showing 1 changed file with 246 additions and 0 deletions.
diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,246 @@
+## Running development version of CAPT using existing Tinkerbell instance
+
+If you have Tinkerbell running in your environment, you can use it for CAPT.
+
+### Requirements
+
+Here is the list of required components to use CAPT:
+
+- Existing Tinkerbell installation running at least versions mentioned in [sandbox](https://github.com/tinkerbell/sandbox/commit/be40a7b371fdab947a413e190cfe99b692651d8e).
+- A Kubernetes cluster which pods has access to your Tinkerbell instance.
+- A Container registry reachable by both your Kubernetes cluster and your workstation with
+  write access.
+- At least one Hardware available with DHCP IP address configured on first interface and with
+  /dev/vda disk.
+- `git` binary
+- `tilt` binary
+- `kubectl` binary
+- `clusterctl` binary
+- `go` binary
+
+### Workaround for 8000 bytes template limit
+
+CAPT creates rather large templates for machine provisioning, so with regular Tinkerbell
+installation you will probably hit this [limit](https://github.com/tinkerbell/tink/issues).
+
+To workaround that, run the following SQL command in your Tinkerbell database:
+```sql
+drop trigger events_channel ON events;
+```
+
+**WARNING: This will disable events streaming feature!**
+
+If you use Tinkerbell [sandbox](https://github.com/tinkerbell/sandbox), you can run the following command
+in your `deploy/` directory:
+```sh
+PGPASSWORD=tinkerbell docker-compose exec db psql -U tinkerbell -c 'drop trigger events_channel ON events;'
+```
+
+### Provisioning image
+
+CAPT requires `ubuntu-install` Docker image to provision Kubernetes machines. You should build this image
+yourself from the `Dockerfile` below and push it to your Tinkerbell container registry.
+
+```
+FROM alpine:3.13
+
+RUN apk add -U qemu-img
+```
+
+You can build and push this image using example commands:
+```sh
+docker build -t 10.17.3.2/ubuntu-install .
+docker push 10.17.3.2/ubuntu-install
+```
+
+### Deploying CAPT
+
+To run CAPT, we're going to use `tilt`.
+
+First, make sure your `kubeconfig` points to the right Kubernetes cluster.
+
+Take a note of name of your `kubeconfig` context, which will be used in Tilt configuration.
+
+You can get current context name using the following command:
+```sh
+kubectl config get-contexts  | awk '/^*/ {print $2}'
+```
+
+Then, run the following commands to clone code we're going to run:
+```sh
+git clone https://github.com/kubernetes-sigs/cluster-api
+git clone [email protected]:tinkerbell/cluster-api-provider-tink.git
+cd cluster-api-provider-tink
+git fetch origin 71d3cae81423b46214f9a303510ee0e2209c37bb
+git checkout 71d3cae81423b46214f9a303510ee0e2209c37bb
+cd ../cluster-api
+git checkout release-0.3
+```
+
+Now, create a configuration file for Tilt. You can run the command below to create a sample config file,
+then replace placeholders with actual values:
+```sh
+cat <<EOF > tilt-settings.json
+{
+  "default_registry": "quay.io/<your username>",
+  "provider_repos": ["../cluster-api-provider-tink"],
+  "enable_providers": ["tinkerbell", "docker", "kubeadm-bootstrap", "kubeadm-control-plane"],
+  "allowed_contexts": ["<your kubeconfig context to use"]
+}
+EOF
+```
+
+Now, export information about your Tinkerbell instance:
+```sh
+export TINKERBELL_GRPC_AUTHORITY=10.17.3.2:42113 TINKERBELL_CERT_URL=http://10.17.3.2:42114/cert
+```
+
+**NOTE: The IP addresses above must be reachable from Pods running on your Kubernetes cluster.**
+
+Finally, run Tilt to deploy CAPI and CAPT to your cluster using the command below:
+```sh
+tilt up
+```
+
+You can now open a webpage printed by Tilt to see the progress on the deployment.
+
+### Generating infrastructure manifests and configuring clusterctl
+
+As CAPT is not yet added in CAPI registry and has no releases yet, we must generate required manifests manually.
+
+To do that, go to cloned `cluster-api-provider-tink` repository from previous step and run the following command:
+```sh
+make release-manifests
+```
+
+It should create `out/release/infrastructure-tinkerbell/v0.0.0-dirty/infrastructure-components.yaml` file for you.
+
+Now, we need to refer this file in `clusterctl` configuration.
+
+If you don't have `clusterctl` configuration yet, you can create one using the following command:
+```sh
+mkdir ~/.cluster-api
+cat <<EOF > ~/.cluster-api/clusterctl.yml
+providers:
+  - name: tinkerbell
+    url: "file://$(realpath out/release/infrastructure-tinkerbell/v0.0.0-dirty/infrastructure-components.yaml)"
+    type: InfrastructureProvider
+EOF
+```
+
+If you already have a configuration file, then add snippet generated by the command below to your `providers` list:
+```sh
+cat <<EOF
+  - name: tinkerbell
+    url: "file://$(realpath out/release/infrastructure-tinkerbell/v0.0.0-dirty/infrastructure-components.yaml)"
+    type: InfrastructureProvider
+EOF
+```
+
+### Adding Hardware objects to your cluster
+
+Before we create a workload cluster, we must register some of Tinkerbell Hardware in Kubernetes to make it visible
+by CAPT controllers.
+
+List hardware you have available in Tinkerbell e.g. using `tink hardware list` command and save Hardware UUIDs which
+you would like to use.
+
+Then, create similar YAML file, which we later apply on the cluster:
+```yaml
+kind: Hardware
+apiVersion: tinkerbell.org/v1alpha1
+metadata:
+  name: first-hardware
+spec:
+  id: <put hardware ID here>
+---
+kind: Hardware
+apiVersion: tinkerbell.org/v1alpha1
+metadata:
+  name: second-hardware
+spec:
+  id: <put hardware ID here>
+```
+
+Now, apply created YAML file on your cluster.
+
+At least one Hardware is required to create a controlplane machine. This guide uses 2 Hardwares, one for controlplane
+machine and one for worker machine.
+
+**NOTE: CAPT expects Hardware to have DHCP IP address configured on first interface of the Hardware. This IP will
+be then used for Node Internal IP.**
+
+To confirm that your Hardware entries are correct, run the following command:
+```sh
+kubectl describe hardware
+```
+
+In the output, you should be able to find MAC address and IP addresses of the hardware.
+
+**WARNING: CAPT currently expects hardwares to have /dev/vda disk, where OS will be installed!**
+
+### Creating workload clusters
+
+With all the steps above, we can now create a workload cluster.
+
+So, let's start with generating the configuration for your cluster using the command below:
+```sh
+clusterctl config cluster capi-quickstart --infrastructure=tinkerbell:v0.0.0-dirty --kubernetes-version=v1.20.0 --control-plane-machine-count=1 --worker-machine-count=1 > test-cluster.yaml
+```
+
+Inspect the new configuration generated in `test-cluster.yaml` and modify it as needed.
+
+Finally, run the following command to create a cluster:
+```sh
+kubectl apply -f test-cluster.yaml
+```
+
+### Observing clsuter provisioning
+
+Few seconds after creating a workload cluster, you should see some log messages in Tilt tab with CAPT that IP address has been selected for controlplane machine etc.
+
+If you list your Hardwares now with labels, you should see which Hardware has been selected by CAPT controllers:
+```sh
+kubectl get hardwares --show-labels
+```
+
+You should also be able to list and describe the created workflows for machine provisioning using the commands below:
+```sh
+kubectl get workflows
+kubectl describe workflows
+```
+
+Once workflows are created, make sure your machines boot from the network to pick up new Workflow.
+
+In the output of commands above, you can see status of provisioning workflows. If everything goes well, reboot step should be the last step you can see.
+
+You can also check general cluster provisioning status using the commands below:
+```sh
+kubectl get kubeadmcontrolplanes
+kubectl get machines
+```
+
+### Getting access to workload cluster
+
+To finish cluster provisioning, we must get access to it and install a CNI plugin. In this guide we will use Calico.
+
+Run the following command to fetch `kubeconfig` for your workload cluster:
+```sh
+clusterctl get kubeconfig capi-quickstart > kubeconfig-workload
+```
+
+Now you can apply Calico using the command below:
+```sh
+KUBECONFIG=kubeconfig-workload kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
+```
+
+At this point your workload cluster should be ready for other deployments.
+
+### Cleaning up
+
+To remove created cluster resources, run the following command:
+```sh
+kubectl delete -f test-cluster.yaml
+```
+
+Right now CAPT does not de-provision the hardware when cluster is removed but makes Hardware available again for other clusters. To make sure machines can be provisioned again, securely wipe their disk and reboot them.