Setup Cluster

[Jan-Willem]

Experimental 4-Node Kubernetes Deployment

Requirements

• A physical 4-node deployment:
• 1 main (control plane) node
• 3 worker nodes
• Each node is equipped with local storage
• Deployment is not required to have optimized performance but must work within the specs of the components used.
• Make use, as far as possible, of hardware that we already have.
• Kubernetes deployed
• Documented and reproducible setup ( Infrastructure as Code / IaC )

Definition of Done

• Hardware and Software Selection

  • Develop a list of hardware components #20
  • Document software choices with rationale #21

• Physical Infrastructure Preparation

  • Install local storage on each node #22
  • Physically mount servers in racks
  • Configure and test network interconnects #23

• Software Deployment

  • Select and install the base operating system
  • Server role into Ansible #24
  • Deploy Kubernetes #25
  • Validate node functionality

• Observability

  • Have the ability to monitor and understand the health and performance of a Kubernetes cluster and its components. #19

• Documentation (project wiki) #26

  • Create a setup procedure
  • Include troubleshooting guide
  • Document network and hardware configuration

Key Decision Points to Discuss

1. Interconnect Technology

   • Evaluate options (e.g., Ethernet, InfiniBand)

2. Operating System Selection

   • Compare RedHat Enterprise Linux vs. Ubuntu Server
   • Evaluate support, stability, and Kubernetes compatibility

3. Configuration Management

   • Assess Ansible for automated deployment
   • Compare with alternative configuration management tools

4. Kubernetes Distribution

   • https://blog.cloudowski.com/articles/which-kubernetes-for-on-prem/
   • Rancher, k3s, OpenShift, OKD, Tanzu, ...

5. Observability/LMA

   • https://github.com/prometheus-operator/kube-prometheus
   • kube-prometheus, VictoriaMetrics, Vector, InfluxDB

Artifacts

• Fully functional 4-node Kubernetes deployment
• Comprehensive setup documentation / Infra as Code (IaC)
• Architectural decision record (ADR)

Additional Considerations

• Ensure scalability for future expansion
• Plan for potential future performance optimizations
• Consider security best practices during setup

Success Criteria

• Deployment must be:
  • Fully operational
  • Reproducible
  • Documented
  • Capable of basic Kubernetes workload deployment

[Jan-Willem]

From today's discussion, the R620s in Charlottesville will be used for the cluster. @daachi will check whether we have the necessary interconnects and hard drives in Charlottesville.

[daachi]

We have 4 R620s racked up. I will begin building them out, but this work will be covered in separate tickets.