diff --git a/docs/README.md b/docs/README.md index 8c19946a2..9db1207e2 100644 --- a/docs/README.md +++ b/docs/README.md @@ -9,6 +9,8 @@ Execute these commands from the `/docs/` directory. ### Installation +From the `vcluster/docs` directory, execute: + ``` $ yarn ``` diff --git a/docs/pages/fragments/deploy-vcluster.mdx b/docs/pages/fragments/deploy-vcluster.mdx deleted file mode 100644 index e52595b3a..000000000 --- a/docs/pages/fragments/deploy-vcluster.mdx +++ /dev/null @@ -1,58 +0,0 @@ -import Tabs from '@theme/Tabs' -import TabItem from '@theme/TabItem' - - - - -```bash -# Create a new vCluster in namespace vCluster-my-vcluster -vcluster create my-vcluster - -# OR: Use --expose to create a vCluster in a remote cluster with an externally accessible LoadBalancer -vcluster create my-vcluster --expose - -# OR: Use -f to use an additional helm values.yaml with extra chart options to deploy vCluster -vcluster create my-vcluster -f values.yaml - -# OR: Use --distro to specify either k0s or vanilla k8s as backing virtual cluster -vcluster create my-vcluster --distro k8s - -# OR: Use --isolate to create an isolated environment for the vCluster workloads -vcluster create my-vcluster --isolate -``` - - - - -Create file `vcluster.yaml`: -```yaml -vcluster: - image: rancher/k3s:v1.23.5-k3s1 # Choose k3s version -``` - -Then, install helm chart using `vcluster.yaml` for chart values: -```bash -helm upgrade --install my-vcluster vcluster \ - --values vcluster.yaml \ - --repo https://charts.loft.sh \ - --namespace vcluster-my-vcluster \ - --repository-config='' -``` - - - - -```bash -kubectl create namespace vcluster-my-vcluster -helm template my-vcluster vcluster --repo https://charts.loft.sh -n vcluster-my-vcluster | kubectl apply -f - -``` - - - diff --git a/docs/pages/fragments/install/cli.mdx b/docs/pages/fragments/install/cli.mdx index 6dc3a1c3e..b3d1a241a 100644 --- a/docs/pages/fragments/install/cli.mdx +++ b/docs/pages/fragments/install/cli.mdx @@ -10,7 +10,6 @@ import TabItem from "@theme/TabItem"; { label: 'Mac (Silicon/ARM)', value: 'mac-arm', }, { label: 'Linux (AMD)', value: 'linux', }, { label: 'Linux (ARM)', value: 'linux-arm', }, - { label: 'Homebrew Installation', value: 'homebrew', }, { label: 'Windows Powershell', value: 'windows', }, ] }> @@ -20,6 +19,9 @@ import TabItem from "@theme/TabItem"; brew install loft-sh/tap/vcluster ``` +If you installed the CLI using `brew install vcluster`, you should `brew uninstall vcluster` and then install using the tap. +The binaries in the tap are signed using the [Sigstore](https://docs.sigstore.dev/) framework for enhanced security. + @@ -48,13 +50,6 @@ curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/downloa curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-linux-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster ``` - - - -```bash -brew install vcluster -``` - diff --git a/docs/pages/getting-started/cleanup.mdx b/docs/pages/getting-started/cleanup.mdx index be070fc42..fb23f25d7 100644 --- a/docs/pages/getting-started/cleanup.mdx +++ b/docs/pages/getting-started/cleanup.mdx @@ -1,5 +1,5 @@ --- -title: Delete vClusters +title: Delete vCluster sidebar_label: 4. Cleanup --- @@ -8,5 +8,5 @@ import DeleteFragment from '../fragments/delete-vcluster.mdx' :::caution Resources inside vClusters -Deleting a vCluster will also delete all objects within and all states related to the vCluster. +Deleting a vCluster also delete all objects within and all states related to the vCluster. ::: diff --git a/docs/pages/getting-started/connect.mdx b/docs/pages/getting-started/connect.mdx index 022d05c01..1dc9eefd3 100644 --- a/docs/pages/getting-started/connect.mdx +++ b/docs/pages/getting-started/connect.mdx @@ -1,117 +1,137 @@ --- -title: Connect to and use vCluster +title: Connect to and Use vCluster sidebar_label: 3. Use vCluster --- -Now that we deployed a vCluster, let's connect to it, run a couple of `kubectl` commands inside of it and then understand what happens behind the scenes inside our vCluster's host namespace that is part of the underlying host cluster. +## Learning objectives -## Connection to the vCluster +1. [Connect to your vCluster instance](#connect-to-your-vcluster). +1. [Run some `kubectl` commands inside of it](#run-kubectl-commands). +1. [Learn what happens](#what-happens-in-the-host-cluster) behind the scenes inside your vCluster's host namespace, which is part of the underlying host cluster. -By default, vCluster CLI will connect to the virtual cluster either directly (on local Kubernetes distributions) or via port-forwarding for remote clusters. +## Connect to your vCluster -If you want to use vCluster without port-forwarding, you can take a look at [other supported exposing methods](../using-vclusters/access.mdx). +To connect to your cluster, run `vcluster connect my-cluster`. Output is similar to: + +```bash +done Switched active kube context to vcluster_my-cluster +- Use `vcluster disconnect` to return to your previous kube context +``` + +By default, the vCluster CLI connects to the virtual cluster either directly (on local Kubernetes distributions) or via port-forwarding for remote clusters. If you want to use vCluster on remote clusters without port-forwarding, you can take a look at [other supported exposing methods](../using-vclusters/access.mdx). ## Run kubectl commands -A virtual cluster behaves the same way as a regular Kubernetes cluster. That means you can run any `kubectl` command and since you are admin of this vCluster, you can even run commands like these: +A virtual cluster behaves the same way as a regular Kubernetes cluster. That means you can run any `kubectl` command. Since you are admin of this vCluster, you can even run commands like these: + ```bash kubectl get namespace kubectl get pods -n kube-system ``` -Let's create a namespace and a demo nginx deployment to understand how vClusters work: +## What happens in the host cluster + +To illustrate what happens in the host cluster, create a namespace and deploy NGINX: + ```bash kubectl create namespace demo-nginx -kubectl create deployment nginx-deployment -n demo-nginx --image=nginx +kubectl create deployment nginx-deployment -n demo-nginx --image=nginx -r 2 ``` -You can check that this demo deployment will create pods inside the vCluster: +Check that this deployment creates 2 pods inside the virtual cluster: + ```bash kubectl get pods -n demo-nginx ``` -## What happens in the host cluster? -The first thing to understand is that **most** resources inside your vCluster will only exist in your vCluster and **not** make it to the underlying host cluster / host namespace. +Output is similar to: -### 1. Use Host Cluster Kube-Context -Let's verify this and switch our kube-context back to the host cluster: ```bash -# Switch back to the host cluster -vcluster disconnect +NAME READY STATUS RESTARTS AGE +nginx-deployment-6d6565499c-2wfrd 1/1 Running 0 9s +nginx-deployment-6d6565499c-2blwr 1/1 Running 0 9s ``` -### 2. Check Namespaces -Now, let's check the namespaces in our host cluster: -```bash -kubectl get namespaces -``` -```bash {3} -NAME STATUS AGE -default Active 11d -vcluster-my-vcluster Active 9m17s -kube-node-lease Active 11d -kube-public Active 11d -kube-system Active 11d -``` -You will notice that there is **no namespace `demo-nginx`** because this namespace only exists inside the vCluster. Everything that belongs to the vCluster will always remain inside the vCluster's host namespace `vcluster-my-vcluster`. +**Most** resources inside your virtual cluster only exist in your virtual cluster and **not** in the underlying host cluster / host namespace. -### 3. Check Deployments -So, let's check to see if our deployment `nginx-deployment` has made it to the underlying host cluster: -```bash -kubectl get deployments -n vcluster-my-vcluster -``` -```bash -No resources found in vCluster-my-vcluster namespace. -``` +To verify this, perform these steps: + +1. Switch back to the host context. + + ```bash + vcluster disconnect + ``` + +1. Check namespaces in the host cluster. + + ```bash + kubectl get namespaces + ``` + + Output is similar to: + + ```bash {3} + NAME STATUS AGE + default Active 11d + vcluster-my-vcluster Active 9m17s + kube-node-lease Active 11d + kube-public Active 11d + kube-system Active 11d + ``` + + Notice that there is **no namespace `demo-nginx`** because this namespace only exists inside the virtual cluster. + + Everything that belongs to the virtual cluster always remains inside the vCluster's `vcluster-my-vcluster` namespace. + +1. Look for the NGINX deployment. + + Check to see if your deployment `nginx-deployment` is in the underlying host cluster. + + ```bash + kubectl get deployments -n vcluster-my-cluster + ``` + + Output is similar to: + + ```bash + No resources found in vcluster-my-cluster namespace. + ``` + + You see that there is no deployment `nginx-deployment` because that deployment only lives inside the virtual cluster. + +1. Look for the NGINX pods. + + The last thing to check is pods running inside the virtual cluster namespace: + + ```bash + kubectl get pods -n vcluster-my-cluster + ``` + + Output is similar to: + + ```bash {4-5} + NAME READY STATUS RESTARTS AGE + coredns-68bdd584b4-9n8c4-x-kube-system-x-my-cluster 1/1 Running 0 129m + my-cluster-0 1/1 Running 0 129m + nginx-deployment-6d6565499c-2blwr-x-demo-nginx-x-my-cluster 1/1 Running 0 7m25s + nginx-deployment-6d6565499c-2wfrd-x-demo-nginx-x-my-cluster 1/1 Running 0 7m25s + ``` + + :::info Renaming + As you see in lines 4-5 of the output, the pod name is rewritten during the sync process since vCluster is mapping pods from namespaces inside the virtual cluster into one single host namespace in the underlying host cluster. + ::: + + The vCluster `my-cluster-0` pod contains the virtual cluster’s API server and some additional tools. There’s also a CoreDNS pod, which vCluster uses, and the two NGINX pods. + + The host cluster has the `nginx-deployment` pods because the virtual cluster **does not** have separate nodes or a scheduler. Instead, the virtual cluster has a _syncer_ that synchronizes resources from the virtual cluster to the underlying host namespace. + The vCluster syncer process tells the underlying cluster to schedule workloads. This syncer process communicates with the API server of the host cluster to schedule the pods and keep track of their state. + To prevent collisions, vCluster appends the name of the virtual cluster namespace the pods are running in and the name of the virtual cluster. + + Only very few resources and API server requests actually reach the underlying Kubernetes API server. Only workload-related resources (e.g. Pod) and networking-related resources (e.g. Service) need to be synchronized down to the host cluster since the vCluster does **not** have any nodes or network itself. + + The state of most objects running in the virtual cluster is stored in a database inside it. vCluster uses SQLite by default for that DB, but it can also use etcd or a few other options like PostgreSQL. But pods are scheduled in the host cluster. -You will see that there is **no deployment `nginx-deployment`** because it also just lives inside the virtual cluster. -### 4. Check Pods -The last thing to check is pods: -```bash -kubectl get pods -n vcluster-my-vcluster -``` -```bash {3} -NAME READY STATUS RESTARTS AGE -coredns-66c464876b-p275l-x-kube-system-x-my-vcluster 1/1 Running 0 14m -nginx-deployment-84cd76b964-mnvzz-x-demo-nginx-x-my-vcluster 1/1 Running 0 10m -my-vcluster-0 2/2 Running 0 14m -``` -And there it is! The pod that has been scheduled for our `nginx-deployment` has actually made it to the underlying host cluster. - -The reason for this is that vClusters do **not** have separate nodes\*. Instead, they have a **syncer** which synchronizes resources from the vCluster to the underlying host namespace to actually get the pods of the vCluster running on the host cluster's nodes and the containers started inside the underlying host namespace. - -:::info Renaming -As you can see above in line 3, the names of pods get rewritten during the sync process since we are mapping pods from X namespaces inside the vCluster into one single host namespace in the underlying host cluster. -::: - -## Benefits of Virtual Clusters -Virtual clusters provide immense benefits for large-scale Kubernetes deployments and multi-tenancy: -- **Full Admin Access**: - - Deploy operators with CRDs, create namespaces and other cluster scoped resources that's usually not possible in namespaces. - - Taint and label nodes without any influence on the host cluster. - - Reuse and share services across multiple virtual clusters with ease. -- **Cost Savings:** - - You can create lightweight vClusters that share the underlying host cluster instead of creating separate "real" clusters. - - vClusters are just deployments, so they can be easily auto-scaled, purged, snapshotted and moved. -- **Low Overhead:** - - vClusters are super lightweight and only reside in a single namespace. - - vClusters run with k3s, a super low-footprint k8s distribution, but they can also run with "real" k8s. - - The control plane of a vCluster runs inside a single pod (+1 CoreDNS pod for vCluster-internal DNS capabilities). -- **No Network Degradation:** - - Since the pods and services inside a vCluster are actually being synchronized down to the host cluster\*, they are effectively using the underlying cluster's pod and service networking and are therefore not a bit slower than any other pods in the underlying host cluster. -- **API Server Compatibility:** - - vClusters run with the k3s API server which is certified k8s distro which ensures 100% Kubernetes API server compliance. - - vCluster have their own API server, controller-manager and a separate, isolated data store (sqlite for easiest option but this is configurable, you can also deploy a full-blown etcd if needed). -- **Security:** - - vCluster users need much fewer permissions in the underlying host cluster / host namespace. - - vCluster users can manage their own CRDs independently and can even mess with RBAC inside their own vClusters. - - vClusters provide an extra layer of isolation because each vCluster has its own API server and control plane (much fewer requests to the underlying cluster that need to be secured\*). -- **Scalability:** - - Less pressure / fewer requests on the k8s API server in large-scale cluster\* - - Higher scalability of cluster via cluster sharding / API server sharding into smaller vClusters - - No need for cluster admins to worry about conflicting CRDs or CRD versions with growing number of users and deployments - -\* Only very few resources and API server requests actually reach the underlying Kubernetes API server. Only workload-related resources (e.g. Pod) and networking-related resources (e.g. Service) need to be synchronized down to the host cluster since the vCluster does **not** have any nodes or network itself. + diff --git a/docs/pages/getting-started/deployment.mdx b/docs/pages/getting-started/deployment.mdx index 5726e92e8..8922346ce 100644 --- a/docs/pages/getting-started/deployment.mdx +++ b/docs/pages/getting-started/deployment.mdx @@ -1,30 +1,117 @@ --- title: Deploy vCluster -sidebar_label: 2. Deploy vClusters +sidebar_label: 2. Deploy vCluster --- +import Tabs from '@theme/Tabs' +import TabItem from '@theme/TabItem' -import DeploySegment from '../fragments/deploy-vcluster.mdx' -Let's create a virtual cluster `my-vcluster` in namespace `vcluster-my-vcluster`: +## Before you begin - +### Admin permissions not required + +One of the biggest benefits of vCluster compared to other virtual cluster approaches is that it does not require any special permissions. You can spin up a virtual cluster even if you do not have a cluster-admin role and only have access to deploy applications to one specific namespace. + +### Namespaces + +Installation creates the namespace for vCluster if it does not yet exist. When you create namespaces within your virtual cluster, those resources are encapsulated within the vCluster namespace. + +## Create your virtual cluster + +Create a virtual cluster `my-vcluster` in namespace `vcluster-my-vcluster`. + + + + +Execute `vcluster create my-cluster` to create a new vCluster instance in namespace `vcluster-my-vcluster`. [K3s](https://k3s.io/) is the default Kubernetes distribution. + +When the installation finishes, you should see output similar to: + +```bash +done Switched active kube context to vcluster_my-cluster +- Use `vcluster disconnect` to return to your previous kube context +``` + +`vcluster create` has [config options](../cli/vcluster_create.md) for specific cases: + +* Use `--expose` to create a vCluster in a remote cluster with an externally accessible LoadBalancer. + + `vcluster create my-vcluster --expose` + +* Use `-f` to use an additional Helm `values.yaml` with extra chart options to deploy vCluster. + + `vcluster create my-vcluster -f values.yaml` + +* Use `--distro` to specify either k0s or vanilla k8s as a backing virtual cluster. + + `vcluster create my-vcluster --distro k8s` + +* Use `--isolate` to create an isolated environment for the vCluster workloads + + `vcluster create my-vcluster --isolate` + + + + + + +You want vCluster to use K3s, so create a file called `vcluster.yaml` with the following contents: + +```yaml +vcluster: + image: rancher/k3s:v1.29.1+k3s2 # Choose K3s version +``` + +Then, install the Helm chart, specifying `vcluster.yaml` for chart values: + +```bash +helm upgrade --install my-vcluster vcluster \ + --values vcluster.yaml \ + --repo https://charts.loft.sh \ + --namespace vcluster-my-vcluster \ + --repository-config='' +``` + + + + +This installs vCluster with default values. + +```bash +kubectl create namespace vcluster-my-vcluster +helm template my-vcluster vcluster --repo https://charts.loft.sh -n vcluster-my-vcluster | kubectl apply -f - +``` + + + + +:::note Contexts +In this guide, vCluster installation creates a new context that starts with "vcluster_my-cluster" and updates your `kubeconfig` file to point to that context. + +* Execute `vcluster connect my-cluster` to connect to your vCluster context. +* Execute `vcluster disconnect` to switch back to your default (host) context. -:::info Host Namespace -The host namespace will be generated if it does not yet exist. No matter which namespaces we create within the virtual cluster, everything will be encapsulated inside the created host namespace that the virtual cluster runs inside. ::: -:::caution Air-gapped Clusters -If you want to deploy vClusters in an air-gapped environment, you can set the following option in the `values.yaml` used to deploy vCluster: + +## Air-Gapped installation + +If you want to deploy vCluster in an air-gapped environment, set the following option in the `values.yaml` used to deploy vCluster: + ``` defaultImageRegistry: my-private-registry:5000/vcluster/ ``` -This will tell vCluster to prepend the above image registry to all images used by vCluster, such as syncer, k3s, coredns etc. So for example `rancher/k3s:v1.22.2-k3s1` will become `my-private-registry:5000/vcluster/rancher/k3s:v1.22.2-k3s1` -You can find a list of all needed images by vCluster in the file `vcluster-images.txt` at the [releases page](https://github.com/loft-sh/vcluster/releases), as well as two scripts (download-images.sh & push-images.sh) to pull and push those to your private registry. -::: +This tells vCluster to prepend the image registry to all images used by vCluster, such as syncer, K3s, and CoreDNS. For example, `rancher/k3s:v1.29.1+k3s2` becomes `my-private-registry:5000/vcluster/rancher/k3s:v1.29.1+k3s2. +You can find a list of all needed images by vCluster in the file `vcluster-images.txt` at the [releases page](https://github.com/loft-sh/vcluster/releases), as well as two scripts (download-images.sh & push-images.sh) to pull and push those to your private registry. -## Admin Permissions Required? -One of the biggest benefits of vCluster compared to other virtual cluster approaches is that it does not require any special permissions. Even if you are not cluster-admin and only have access to deploy applications to one specific namespace, you will very likely be able to spin up a virtual cluster. +You can locate the Helm chart and values file for Kubernetes distro in the vCluster [repo](https://github.com/loft-sh/vcluster/tree/main/charts). Be sure to choose the tag that matches your vCluster version. -Check out the `kubectl` tab above to see what `vcluster create` is actually deploying to the host-namespace. diff --git a/docs/pages/getting-started/setup.mdx b/docs/pages/getting-started/setup.mdx index 91a2155c4..bc5707419 100644 --- a/docs/pages/getting-started/setup.mdx +++ b/docs/pages/getting-started/setup.mdx @@ -1,22 +1,41 @@ --- -title: Install vCluster CLI +title: Install the vCluster CLI sidebar_label: 1. Download CLI --- import InstallCLIFragment from '../fragments/install/cli.mdx' -:::note About This Guide -**Goal**: Install vCluster CLI, deploy virtual clusters and learn how they work -**Estimated time**: 10 minutes -**Requirements:** -- `kubectl` (check via `kubectl version`) -- `helm` v3 (check with `helm version`) -- a working kube-context with access to a Kubernetes cluster (check with `kubectl get namespaces`) -::: +## Learning objectives +In this guide, you learn the following: -## Download vCluster CLI -Use one of the following commands to download the vCluster CLI binary from GitHub: +1. How to install the vCluster CLI. +1. [How to deploy vCluster](./deployment.mdx). +1. [How to connect to vCluster and how vCluster interacts with the host cluster](./connect.mdx). +1. [How to uninstall vCluster](./cleanup.mdx). + +These exercises should take about 10 minutes. + +You can join the [Loft Users Slack](https://slack.loft.sh/) to ask questions in the #vcluster channel. + +Terms: + +* **vCluster**: the product; this also refers to a vCluster installation. +* `vcluster`: the CLI. +* **host**: The Kubernetes cluster that you install vCluster on. + + +## Before you begin + +- You have read [What Are Virtual Clusters](../what-are-virtual-clusters.mdx). +- You have installed [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) (check via `kubectl version`) and [Helm v3](https://helm.sh/docs/intro/install/) (check with `helm version`). +- You have access to a Kubernetes cluster to use as the host cluster (check with `kubectl get namespaces`) + +## Install the vCluster CLI + +Use one of the following commands to download and install the vCluster CLI binary from GitHub: + + diff --git a/docs/pages/what-are-virtual-clusters.mdx b/docs/pages/what-are-virtual-clusters.mdx index 85241477d..3d5046371 100644 --- a/docs/pages/what-are-virtual-clusters.mdx +++ b/docs/pages/what-are-virtual-clusters.mdx @@ -46,3 +46,31 @@ Finally, virtual clusters can be configured independently of the physical cluste
vCluster - Comparison
+## Benefits of vClusters + +vClusters provide immense benefits for large-scale Kubernetes deployments and multi-tenancy. + +- **Full Admin Access**: + - Deploy operators with CRDs, create namespaces and other cluster-scoped resources that you normally can't create inside a namespace. + - Taint and label nodes without influencing the host cluster. See [Nodes](./architecture/nodes.mdx) and [Pod Scheduling](./architecture/scheduling.mdx) for more information. + - Reuse and share services across multiple virtual clusters with ease. +- **Cost Savings:** + - Create lightweight vClusters that share the underlying host cluster instead of creating separate "real" clusters. + - Auto-scale, purge, snapshot, and move your vClusters, since they are Kubernetes deployments. +- **Low Overhead:** + - vClusters are super lightweight and only reside in a single namespace. + - vClusters run with [K3s](https://k3s.io/), a super low-footprint K8s distribution. You can use other [supported distributions](./deploying-vclusters/supported-distros) such as [K0s](https://k0sproject.io/), vanilla [Kubernetes](https://kubernetes.io/), and [AWS EKS](https://aws.amazon.com/eks/). + - The vCluster control plane runs inside a single pod. Open source vCluster also uses a CoreDNS pod for vCluster-internal DNS capabilities. With vCluster.Pro, however, you can [enable the integrated CoreDNS](https://www.vcluster.com/pro/docs/features/integrated_coredns) so you don't need the additional pod. +- **No Network Degradation:** + - Since the pods and services inside a vCluster are actually being synchronized down to the host cluster, they are effectively using the underlying cluster's pod and service networking. The vCluster pods are as fast as other pods in the underlying host cluster. +- **API Server Compatibility:** + - vClusters run with the API server from the Kubernetes distribution that you choose to use. This ensures 100% Kubernetes API server compliance. + - vCluster manages its API server, controller-manager, and a separate, isolated data store. Use the embedded SQLite or a full-blown etcd if that's what you need. See [Persisting vCluster data](./deploying-vclusters/persistence) for a list of supported data stores. +- **Security:** + - vCluster users need fewer permissions in the underlying host cluster / host namespace. + - vCluster users can manage their own CRDs independently and can even modify RBAC inside their own vClusters. + - vClusters provide an extra layer of isolation. Each vCluster manages its own API server and control plane, which means that fewer requests to the underlying cluster need to be secured. +- **Scalability:** + - Less pressure / fewer requests on the K8s API server in a large-scale cluster. + - Higher scalability of clusters via cluster sharding / API server sharding into smaller vClusters. + - No need for cluster admins to worry about conflicting CRDs or CRD versions with a growing number of users and deployments. \ No newline at end of file