Skip to content

Latest commit

 

History

History
103 lines (90 loc) · 4.5 KB

K3s_and_K3AI.adoc

File metadata and controls

103 lines (90 loc) · 4.5 KB
Raw

K3AI

  • Command line tool that takes in a specification and deploys a K8s cluster

  • Further invocations can add AI frameworks and tools to the deployed cluster

    • Runs on Windows, Mac, Linux and limited ARM support

    • Can deploy AI to existing cluster

    • Not sure if you can deploy the cluster and specific AI tools at the same time

    • As of 01/2021 can deploy:

      • Rancher K3s to pre-provisioned nodes

      • Rancher K3d to a docker enabled host

      • Mirantis K0s (have no idea how it works)

      • KinD to a docker enabled host

      • A "remote" cluster

K3s

  • Server install command that sets the permissions correctly for the KUBECONFIG file: curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE=644 sh -s -

  • Agent install command (K3S_URL sets the node to worker mode): curl -sfL https://get.k3s.io | K3S_URL=https://<FQDN of server>:6443 K3S_TOKEN=<token> sh -

    • The token is found on the Agent node in /var/lib/rancher/k3s/server/node-token

    • Likely need to provide LB address for the FQDN of the server for HA

      • Though, not if all external to API server access will go through Rancher server

Clean way of deploying HA K3s with embedded etcd:

  • Deploy K3s with --cluster-init to enable embedded etcd:

K3s_VERSION=""      #I.e. "v1.20.4+k3s1"
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=${K3s_VERSION} INSTALL_K3S_EXEC='server --cluster-init --write-kubeconfig-mode=644' sh -s -

  • Install additional servers:

FIRST_SERVER_IP=""      # Private IP preferred, if available
NODE_TOKEN=""           # From the /var/lib/rancher/k3s/server/node-token file on the first server
K3s_VERSION=""          # Match the first of the first server
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=${K3s_VERSION} K3S_URL=https://${FIRST_SERVER_IP}:6443 K3S_TOKEN=${NODE_TOKEN} K3S_KUBECONFIG_MODE="644" INSTALL_K3S_EXEC='server' sh -

* Install additional agents:

FIRST_SERVER_IP="" # Private IP preferred, if available NODE_TOKEN="" # From the /var/lib/rancher/k3s/server/node-token file on the first server K3s_VERSION="" # Match the first of the first server









curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=${K3s_VERSION} K3S_URL=https://${FIRST_SERVER_IP}:6443 K3S_TOKEN=${NODE_TOKEN} K3S_KUBECONFIG_MODE="644" sh -






A proven method of creating an HA K3s cluster:


    

  • 

    Deploy K3s with --cluster-init to enable embedded etcd:
    

    • 









wget https://github.com/k3s-io/k3s/releases/download/v1.20.4%2Bk3s1/k3s
sudo mv k3s /usr/local/bin/
sudo chmod 755 /usr/local/bin/k3s
curl -sfL https://get.k3s.io > install.sh
chmod 755 install.sh
INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC='server --cluster-init --write-kubeconfig-mode=644'  ./install.sh








    

  • 

    Add another server to the cluster
    

    • 









wget https://github.com/k3s-io/k3s/releases/download/v1.20.4%2Bk3s1/k3s
sudo mv k3s /usr/local/bin/
sudo chmod 755 /usr/local/bin/k3s
curl -sfL https://get.k3s.io > install.sh
chmod 755 install.sh
FIRST_SERVER_IP=""
sudo K3S_TOKEN=<from the /var/lib/rancher/k3s/server/node-token file on the first server> INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC='server --server https://${FIRST_SERVER_IP}:6443 --write-kubeconfig-mode=644'  ./install.sh












k3sup






    

  • 

    Had tons of trouble creating a cluster manually. K3sup worked well
    

    • 

  • 

    Download and install with:
    

    • 









curl -sLS https://get.k3sup.dev | sh
sudo install k3sup /usr/local/bin/








Create the custer with the first server(master) node:




k3sup install --ip=10.110.2.0 --tls-san=10.110.2.10 --sudo --user sles --cluster --k3s-channel=stable --merge --local-path=$HOME/.kube/config --context=ha-k3ai








    

  • 

    Equals signs are optional
    

    • 

  • 

    --ip is the server to be installed
    

    • 

  • 

    --tls-san ensures the VIP (currently presented by keepalived) is included in the TLS cert
    

    • 

  • 

    --sudo means to use sudo for the installation since the SSH user won’t be root
    

    • 

  • 

    --user is the SSH user account to use on the target system
    

    • 

  • 

    --cluster tells the first server node set etcd up in cluster mode
    

    • 

  • 

    --merge merges the new cluster context (ha-k3ai in this case) with the existing one in ~/.kube/config
    

    

      

    • 

      Note that the KUBECONFIG file won’t materialize if there is no file at that location or it can’t be parsed correctly
.Adding the each additional server(master) node:
      

      • 

    

    

    • 









k3sup join --ip 10.110.2.1 --server --server-ip 10.110.2.10 --k3s-channel stable








    

  • 

    --ip is the server to be installed
    

    • 

  • 

    --server-ip is the VIP for the K3s API server
.Adding each agent(worker) node:
    

    • 









k3sup join --ip 10.110.3.0 --server-ip 10.110.2.10  --sudo --user sles --k3s-channel stable