Provisioning and managing cloud infrastructure and applications can sometimes be daunting for an end-user (and for the platform team). This project is a PoC that addresses the problem by creating an internal developer platform to simplify the process of requesting and managing cloud infrastructure and applications. It's a set of self-served services built by the platform team that supports and accelerates development or analysis, while taking care of managing the underlying infrastructure.
Building a centralized platform for infrastructure helps in achieving a separation of responsibilities between the platform team and the application / analytical teams. For example, an analytical team can request for a Vertex AI Notebook and a Cloud Storage bucket without worrying about the networking setup for these resources via the platform's predefined templates. Such templates also help in driving software development to follow best / standard practices or comply with the Enterprise Architecture Framework, all while lowering the barrier to entry. In addition to this, it can serve as a single pane of glass to monitor and manage cloud costs via a FinOps monitor, host technical documentations via TechDocs, find the information you're looking for throughout the entire ecosystem via it's rich search functionality, track project resources and the relevant metadata (ownership, links to code repositories, etc.) and more...
The portal is built on top of modern cloud-native technological blocks like microservices, declarative APIs, containers, and service meshes. It uses Kubernetes (GKE Autopilot) as the platform and CNCF tools like:
- Flux for GitOps
- Istio for secure networking and observability
- Cert-Manager for automatic renewal of TLS certificates
- Crossplane for managing infrastructure
- Backstage for building the frontend of the platform
It aims to satisfy the following core componenets of an Internal developer platform:
| Core Component | Short Description | Solution |
|---|---|---|
| Application Configuration Management | Manage application configuration in a dynamic, scalable and reliable way. | YAML versioned with Git / Github |
| Infrastructure Orchestration | Orchestrate your infrastructure in a dynamic and intelligent way depending on the context. | Crossplane |
| Environment Management | Enable developers to create new and fully provisioned environments whenever needed. | Depends on what "Environment" means for an application |
| Deployment Management | Implement a delivery pipeline for Continuous Delivery or even Continuous Deployment (CD). | Flux |
| Role-Based Access Control | Manage who can do what in a scalable way. | Backstage Authn / Authz |
TODO
- PoC work was completed and presented internally in the DPI Deep Dive meeting
- Paused until contract goes through to bring on resources