Container deployment

Automation and manifests for container deployment

Index

• Secret Management
• Local setup using Minikube
• Runbook
• Recommended VM requirements
• New environment creation
• Reference docs
• LVM volume setup
• Basic VM Validations
• Datadog Monitoring
• Deploy branch for testing k8s manifest changes
• Infra Requirement for the simple web application
• Manual Changes
• Create New DHIS2 Instance
• DHIS2 Database Restore
• Monitoring
• Terraform
• Kubeprometheus Jsonnet
• Loki
• SSO
• CICD
• AWS
• Kubernetes Upgrade

High-Level Architecture

Components

• Kubernetes: Acts as the container orchestration platform, deploying all application components as containers within a Kubernetes cluster. This setup enables high availability, effective scaling, and seamless management of containerized applications.

• K3s: A lightweight Kubernetes distribution, providing a simplified installation process and reduced resource requirements. K3s is used to deploy Kubernetes clusters on Virtual Machines (VMs).

• Helm: A package manager for Kubernetes that simplifies the deployment and management of applications within the cluster. Helm charts are used to define the desired state of applications, making it easy to install, upgrade, and remove applications in a consistent manner.

• ArgoCD: Facilitates Continuous Delivery (CD) by automating the deployment and synchronization of Kubernetes configurations. Supports various templating engines like Helm, Kustomize, and Jsonnet, making the management of complex deployments straightforward.

• Sealed Secrets: Used for the secure handling of secrets within the Kubernetes environment. Sealed Secrets are encrypted and safely stored in the repository, to be decrypted in the cluster as required, enhancing the security of sensitive information.

• Terraform: Utilizes Infrastructure as Code (IaC) principles to provision and manage AWS Elastic Kubernetes Service (EKS) clusters. This allows for cloud infrastructure to be defined in code, making it easily reproducible and scalable.

• Ansible: Works alongside Terraform to provision Kubernetes clusters on Virtual Machines (VMs). It automates the setup of VMs and the Kubernetes cluster installation, providing flexibility for deployments across different environments.

• Nginx Ingress: Utilized to route external traffic to the Kubernetes cluster, Nginx Ingress serves as a robust, flexible, and efficient HTTP and reverse proxy server, simplifying the exposure of services to the internet.

• Cert-Manager: Automates the management of SSL/TLS certificates, including their creation, renewal, and deployment within the Kubernetes environment. This ensures secure communication and the trusted delivery of content.

• PostgreSQL Cluster Management: Managed by the Crunchy Data Kubernetes Operator, which simplifies the deployment and management of PostgreSQL clusters within Kubernetes. This approach leverages Kubernetes' native capabilities to ensure high availability, performance, and scalability of PostgreSQL databases.

• Reloader: Automatically updates pods in response to config or secret changes. This ensures that applications are always running with the most current configurations and secrets, improving the dynamism and security of deployments.

• Datadog Operator: Responsible for the collection of logs, metrics, and APM data. By deploying the Datadog Operator within the Kubernetes environment, monitoring and observability are seamlessly integrated, providing comprehensive insights into application performance and system health.

• Other Components: In addition to the core components, various other essential services and tools are installed and managed. Please find all the relevant manifests in the k8s/manifests directory of this repository

Folder structure

├── ansible # Ansible automation for k3s cluster, firewall, ssh, etc
├── terraform # Cloud infrastructure provisioning
├── k8s # All k8s related resources manifests
│   ├── environments # Base folder for all environments
│   │   └── <environment-name> # Folder for environment specific manifests and configs
│   |     ├── argocd-apps # Argocd application manifests
|   |     ├── configmaps # Configmaps
|   |     ├── secrets # Sealed secrets files
|   |     ├── op-datadog # Datadog operator manifests
|   |     ├── op-postgres # Postgres operator manifests
|   |     ├── values # Environment specific values files for individual components
|   |     ├── ...
│   │       
│   └── manifests # K8s yaml, helm manifests and wrappers for individual components
│       └── simple-server # Helm chart for simple server
│       |    ├── charts # Packaged dependency
│       |    ├── Chart.yaml # Chart metadata
│       |    ├── templates # Helm template folder
│       |    │   ├── cron.yaml
│       |    │   ├── ingress.yaml
│       |    │   ├── migration-job.yaml
│       |    │   ├── server.yaml
│       |    │   ├── service.yaml
│       |    │   └── worker.yaml
│       |    │   └── ...
│       |    ├── values.bd-k3s-demo.yaml # Override default with demo environment specific values
│       |    ├── values.staging.yaml # Override default with staging environment specific values
│       |    └── values.yaml # Default values
│       |    └── ...
│       ├── argocd # Helm chart for Argocd
│       ├── ingress # Helm chart for Ingress
│       ├── sealed-secrets # Helm chart for Bitnami sealed secrets
│       ├── ...