diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..a8abb0f6 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,246 @@ +## Running development version of CAPT using existing Tinkerbell instance + +If you have Tinkerbell running in your environment, you can use it for CAPT. + +### Requirements + +Here is the list of required components to use CAPT: + +- Existing Tinkerbell installation running at least versions mentioned in [sandbox](https://github.com/tinkerbell/sandbox/commit/be40a7b371fdab947a413e190cfe99b692651d8e). +- A Kubernetes cluster which pods has access to your Tinkerbell instance. +- A Container registry reachable by both your Kubernetes cluster and your workstation with + write access. +- At least one Hardware available with DHCP IP address configured on first interface and with + /dev/vda disk. +- `git` binary +- `tilt` binary +- `kubectl` binary +- `clusterctl` binary +- `go` binary + +### Workaround for 8000 bytes template limit + +CAPT creates rather large templates for machine provisioning, so with regular Tinkerbell +installation you will probably hit this [limit](https://github.com/tinkerbell/tink/issues). + +To workaround that, run the following SQL command in your Tinkerbell database: +```sql +drop trigger events_channel ON events; +``` + +**WARNING: This will disable events streaming feature!** + +If you use Tinkerbell [sandbox](https://github.com/tinkerbell/sandbox), you can run the following command +in your `deploy/` directory: +```sh +PGPASSWORD=tinkerbell docker-compose exec db psql -U tinkerbell -c 'drop trigger events_channel ON events;' +``` + +### Provisioning image + +CAPT requires `ubuntu-install` Docker image to provision Kubernetes machines. You should build this image +yourself from the `Dockerfile` below and push it to your Tinkerbell container registry. + +``` +FROM alpine:3.13 + +RUN apk add -U qemu-img +``` + +You can build and push this image using example commands: +```sh +docker build -t 10.17.3.2/ubuntu-install . +docker push 10.17.3.2/ubuntu-install +``` + +### Deploying CAPT + +To run CAPT, we're going to use `tilt`. + +First, make sure your `kubeconfig` points to the right Kubernetes cluster. + +Take a note of name of your `kubeconfig` context, which will be used in Tilt configuration. + +You can get current context name using the following command: +```sh +kubectl config get-contexts | awk '/^*/ {print $2}' +``` + +Then, run the following commands to clone code we're going to run: +```sh +git clone https://github.com/kubernetes-sigs/cluster-api +git clone git@github.com:tinkerbell/cluster-api-provider-tink.git +cd cluster-api-provider-tink +git fetch origin 71d3cae81423b46214f9a303510ee0e2209c37bb +git checkout 71d3cae81423b46214f9a303510ee0e2209c37bb +cd ../cluster-api +git checkout release-0.3 +``` + +Now, create a configuration file for Tilt. You can run the command below to create a sample config file, +then replace placeholders with actual values: +```sh +cat < tilt-settings.json +{ + "default_registry": "quay.io/", + "provider_repos": ["../cluster-api-provider-tink"], + "enable_providers": ["tinkerbell", "docker", "kubeadm-bootstrap", "kubeadm-control-plane"], + "allowed_contexts": [" ~/.cluster-api/clusterctl.yml +providers: + - name: tinkerbell + url: "file://$(realpath out/release/infrastructure-tinkerbell/v0.0.0-dirty/infrastructure-components.yaml)" + type: InfrastructureProvider +EOF +``` + +If you already have a configuration file, then add snippet generated by the command below to your `providers` list: +```sh +cat < +--- +kind: Hardware +apiVersion: tinkerbell.org/v1alpha1 +metadata: + name: second-hardware +spec: + id: +``` + +Now, apply created YAML file on your cluster. + +At least one Hardware is required to create a controlplane machine. This guide uses 2 Hardwares, one for controlplane +machine and one for worker machine. + +**NOTE: CAPT expects Hardware to have DHCP IP address configured on first interface of the Hardware. This IP will +be then used for Node Internal IP.** + +To confirm that your Hardware entries are correct, run the following command: +```sh +kubectl describe hardware +``` + +In the output, you should be able to find MAC address and IP addresses of the hardware. + +**WARNING: CAPT currently expects hardwares to have /dev/vda disk, where OS will be installed!** + +### Creating workload clusters + +With all the steps above, we can now create a workload cluster. + +So, let's start with generating the configuration for your cluster using the command below: +```sh +clusterctl config cluster capi-quickstart --infrastructure=tinkerbell:v0.0.0-dirty --kubernetes-version=v1.20.0 --control-plane-machine-count=1 --worker-machine-count=1 > test-cluster.yaml +``` + +Inspect the new configuration generated in `test-cluster.yaml` and modify it as needed. + +Finally, run the following command to create a cluster: +```sh +kubectl apply -f test-cluster.yaml +``` + +### Observing clsuter provisioning + +Few seconds after creating a workload cluster, you should see some log messages in Tilt tab with CAPT that IP address has been selected for controlplane machine etc. + +If you list your Hardwares now with labels, you should see which Hardware has been selected by CAPT controllers: +```sh +kubectl get hardwares --show-labels +``` + +You should also be able to list and describe the created workflows for machine provisioning using the commands below: +```sh +kubectl get workflows +kubectl describe workflows +``` + +Once workflows are created, make sure your machines boot from the network to pick up new Workflow. + +In the output of commands above, you can see status of provisioning workflows. If everything goes well, reboot step should be the last step you can see. + +You can also check general cluster provisioning status using the commands below: +```sh +kubectl get kubeadmcontrolplanes +kubectl get machines +``` + +### Getting access to workload cluster + +To finish cluster provisioning, we must get access to it and install a CNI plugin. In this guide we will use Calico. + +Run the following command to fetch `kubeconfig` for your workload cluster: +```sh +clusterctl get kubeconfig capi-quickstart > kubeconfig-workload +``` + +Now you can apply Calico using the command below: +```sh +KUBECONFIG=kubeconfig-workload kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml +``` + +At this point your workload cluster should be ready for other deployments. + +### Cleaning up + +To remove created cluster resources, run the following command: +```sh +kubectl delete -f test-cluster.yaml +``` + +Right now CAPT does not de-provision the hardware when cluster is removed but makes Hardware available again for other clusters. To make sure machines can be provisioned again, securely wipe their disk and reboot them.