diff --git a/PROJECT b/PROJECT index 2bc10b3..0cf8fbe 100644 --- a/PROJECT +++ b/PROJECT @@ -13,4 +13,14 @@ resources: kind: Unstructured path: k8s.io/apimachinery/pkg/apis/meta/v1/unstructured version: v1 +- controller: true + group: core + kind: Pod + path: k8s.io/api/core/v1 + version: v1 +- controller: true + group: core + kind: Service + path: k8s.io/api/core/v1 + version: v1 version: "3" diff --git a/README.md b/README.md index 8988571..19d2cc5 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,23 @@ Scribe is a tool that automates the propagation of annotations across Kubernetes resources based on the annotations in a Namespace. This simplifies the process of managing annotations for multiple resources, ensuring consistency and reducing manual intervention. -## Description +- [Scribe](#scribe) + - [Annotations](#annotations) + - [Prometheus PodMonitors and ServiceMonitors](#prometheus-podmonitors-and-servicemonitors) + - [ServiceMonitors](#servicemonitors) + - [1. Simplest Example](#1-simplest-example) + - [2. Selecting a Specific Port](#2-selecting-a-specific-port) + - [3. Custom Path for Metrics](#3-custom-path-for-metrics) + - [PodMonitors](#podmonitors) + - [1. Simplest Example](#1-simplest-example-1) + - [2. Selecting a Specific Port](#2-selecting-a-specific-port-1) + - [3. Custom Path for Metrics](#3-custom-path-for-metrics-1) + - [Installation](#installation) + - [Contributing](#contributing) + - [End-to-End Tests](#end-to-end-tests) + - [License](#license) + +## Annotations Scribe is designed to streamline the management of Kubernetes annotations by observing namespaces and propagating specific annotations to all resources under its scope, such as Deployments, Pods, or other related objects. For example, you can use Scribe to automatically add annotations to enable features like auto-reload on updates, using tools like Reloader. @@ -30,6 +46,191 @@ metadata: This ensures all resources in the `reloader-example` namespace will inherit the specified annotation, allowing for seamless automation and consistency across deployments. +## Prometheus PodMonitors and ServiceMonitors + +> [!NOTE] +> While using annotations like `prometheus.io/scrape`, `prometheus.io/port`, and `prometheus.io/path` is a convenient way to enable basic metrics scraping, they do not provide the full flexibility and functionality offered by directly managing `ServiceMonitor` or `PodMonitor` resources. +> +> Annotations are limited to straightforward configurations, such as enabling scraping, selecting ports, and specifying a metrics path. However, more advanced use cases - like defining custom scrape intervals, relabeling metrics, or adding TLS configurations - require direct manipulation of `ServiceMonitor` or `PodMonitor` objects. + +One extra feature of Scribe is managing simple monitors from Prometheus. By default, Scribe enables the creation of [`ServiceMonitors`](https://prometheus-operator.dev/docs/api-reference/api/#monitoring.coreos.com/v1.ServiceMonitor) and [`PodMonitors`](https://prometheus-operator.dev/docs/api-reference/api/#monitoring.coreos.com/v1.PodMonitor) to facilitate metrics scraping for Kubernetes resources. This feature can be disabled by adding the annotation `scribe.anza-labs.dev/monitors=disabled` to the resource. + +Scribe leverages common Prometheus annotations (`prometheus.io/*`) to determine the configuration for `ServiceMonitors` and `PodMonitors`. These annotations allow users to define: + +- Whether a resource should be scraped (`prometheus.io/scrape`). +- Specific ports to target (`prometheus.io/port`). +- Custom paths for metrics (`prometheus.io/path`). + +This enables seamless integration with Prometheus while keeping resource configurations simple and intuitive. + +> [!WARNING] +> To successfully create a `ServiceMonitor` or `PodMonitor`, the resource must include at least one label. This is necessary because the `selector.matchLabels` field in the generated monitor uses these labels to identify the corresponding `Service` or `Pod` that Prometheus should scrape. + +### ServiceMonitors + +`ServiceMonitors` are used to scrape metrics from Kubernetes `Service` resources. Scribe automatically generates a `ServiceMonitor` based on the metadata annotations and resource specifications. + +#### 1. Simplest Example + +The following `Service` configuration uses the `prometheus.io/scrape` annotation to enable scraping for all exposed ports. + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: service-monitor-example + labels: + app: example + annotations: + prometheus.io/scrape: 'true' +spec: + selector: + app: example + ports: + - name: web + port: 8080 + targetPort: 80 + - name: web-secure + port: 8443 + targetPort: 443 +``` + +Generated ServiceMonitor: + +```yaml +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + name: service-monitor-example + labels: + app: example +spec: + selector: + matchLabels: + app: example + endpoints: + - port: web + targetPort: 80 + path: '/metrics' + - port: web-secure + targetPort: 443 + path: '/metrics' +``` + +#### 2. Selecting a Specific Port + +If a specific port should be scraped, use the `prometheus.io/port` annotation. It can reference a port number or name. + +Using a port number: + +```yaml +metadata: + annotations: + prometheus.io/scrape: 'true' + prometheus.io/port: '8080' +``` + +Using a port name: + +```yaml +metadata: + annotations: + prometheus.io/scrape: 'true' + prometheus.io/port: 'web-secure' +``` + +#### 3. Custom Path for Metrics + +Define a custom metrics path using the `prometheus.io/path` annotation: + +```yaml +metadata: + annotations: + prometheus.io/scrape: 'true' + prometheus.io/path: '/custom/metrics' +``` + +### PodMonitors + +`PodMonitors` are used to scrape metrics directly from `Pod` resources. Similar to `ServiceMonitors`, they rely on annotations and resource specifications for configuration. + +#### 1. Simplest Example + +A `Pod` exposing multiple ports can use the `prometheus.io/scrape` annotation to enable scraping for all defined ports: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: pod-monitor-example + labels: + app: example + annotations: + prometheus.io/scrape: 'true' +spec: + containers: + - name: nginx + image: nginx:latest + ports: + - name: web + containerPort: 80 + - name: web-secure + containerPort: 443 +``` + +Generated PodMonitor: + +```yaml +apiVersion: monitoring.coreos.com/v1 +kind: PodMonitor +metadata: + name: pod-monitor-example + labels: + app: example +spec: + selector: + matchLabels: + app: example + podMetricsEndpoints: + - port: web + path: '/metrics' + - port: web-secure + path: '/metrics' +``` + +#### 2. Selecting a Specific Port + +To scrape metrics from a specific port, use the `prometheus.io/port` annotation. The port can be specified as a number or name. + +Using a port number: + +```yaml +metadata: + annotations: + prometheus.io/scrape: 'true' + prometheus.io/port: '8080' +``` + +Using a port name: + +```yaml +metadata: + annotations: + prometheus.io/scrape: 'true' + prometheus.io/port: 'web-secure' +``` + +#### 3. Custom Path for Metrics + +Define a custom metrics path using the `prometheus.io/path` annotation: + +```yaml +metadata: + annotations: + prometheus.io/scrape: 'true' + prometheus.io/path: '/custom/metrics' +``` + ## Installation [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/anza-labs)](https://artifacthub.io/packages/search?repo=anza-labs) @@ -50,6 +251,41 @@ To contribute: Make sure to follow the coding standards, and ensure that your code passes all tests and validations. We use `make` to help with common tasks. Run `make help` for more information on all available `make` targets. +Here's a draft for the **End to End Tests** section of your README: + +### End-to-End Tests + +The following steps outline how to run the end-to-end tests for this project. These tests validate the functionality of the system in a fully deployed environment. + +1. **Create the Test Cluster** + Start a local Kubernetes cluster for testing purposes: + + ```sh + make cluster + ``` + +2. **Build and Push the Docker Image** + Build the Docker image and push it to the local registry: + + ```sh + make docker-build docker-push IMG=localhost:5005/manager:e2e + ``` + +3. **Deploy the Application** + Deploy the application using the test image: + + ```sh + make deploy IMG=localhost:5005/manager:e2e + ``` + +After completing these steps, the application will be deployed and ready for end-to-end testing in the local Kubernetes environment. + +The E2E tests can be now run using the following command: + +```sh +make test-e2e +``` + ## License Copyright 2024 anza-labs contributors. @@ -66,4 +302,4 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -[license]: https://github.com/registry-operator/registry-operator/blob/main/LICENSE +[license]: https://github.com/anza-labs/scribe/blob/main/LICENSE diff --git a/cmd/main.go b/cmd/main.go index 9265351..efdd457 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -156,6 +156,30 @@ func main() { os.Exit(1) } } + + promscope, err := controller.NewPrometheusScope(mgr.GetClient(), mgr.GetConfig()) + if err != nil { + setupLog.Error(err, "unable to create scoped client", "scope", "Prometheus") + os.Exit(1) + } + + if err = (&controller.PodReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + PrometheusScope: promscope, + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "Pod") + os.Exit(1) + } + + if err = (&controller.ServiceReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + PrometheusScope: promscope, + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "Service") + os.Exit(1) + } // +kubebuilder:scaffold:builder if err := mgr.AddHealthzCheck("healthz", healthz.Ping); err != nil { diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 392d95f..231bcb3 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -8,7 +8,22 @@ rules: - "" resources: - namespaces + - pods + - services verbs: - get - list - watch +- apiGroups: + - monitoring.coreos.com + resources: + - podmonitors + - servicemonitors + verbs: + - create + - delete + - get + - list + - patch + - update + - watch diff --git a/examples/reloader.yaml b/examples/reloader.yaml index 59bc4f3..7a07317 100644 --- a/examples/reloader.yaml +++ b/examples/reloader.yaml @@ -25,7 +25,7 @@ spec: spec: containers: - name: nginx - image: nginx + image: docker.io/library/nginx:latest resources: limits: memory: "128Mi" diff --git a/go.mod b/go.mod index bff8ebe..5c632ee 100644 --- a/go.mod +++ b/go.mod @@ -5,6 +5,9 @@ go 1.23.0 toolchain go1.23.3 require ( + dario.cat/mergo v1.0.1 + github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.78.2 + github.com/stretchr/testify v1.10.0 k8s.io/api v0.31.3 k8s.io/apimachinery v0.31.3 k8s.io/client-go v0.31.3 @@ -95,7 +98,7 @@ require ( github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c // indirect github.com/docker/go-connections v0.4.0 // indirect github.com/docker/go-metrics v0.0.1 // indirect - github.com/docker/go-units v0.4.0 // indirect + github.com/docker/go-units v0.5.0 // indirect github.com/dustinkirkland/golang-petname v0.0.0-20240428194347-eebcea082ee0 // indirect github.com/emicklei/go-restful/v3 v3.12.1 // indirect github.com/ettle/strcase v0.2.0 // indirect @@ -227,6 +230,7 @@ require ( github.com/olekukonko/tablewriter v0.0.5 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/image-spec v1.0.3-0.20211202183452-c5a74bcca799 // indirect + github.com/opentracing/opentracing-go v1.2.0 // indirect github.com/pelletier/go-toml v1.9.5 // indirect github.com/pelletier/go-toml/v2 v2.2.3 // indirect github.com/peterbourgon/diskv v2.0.1+incompatible // indirect @@ -235,9 +239,9 @@ require ( github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect github.com/polyfloyd/go-errorlint v1.7.0 // indirect github.com/power-devops/perfstat v0.0.0-20220216144756-c35f1ee13d7c // indirect - github.com/prometheus/client_golang v1.20.4 // indirect + github.com/prometheus/client_golang v1.20.5 // indirect github.com/prometheus/client_model v0.6.1 // indirect - github.com/prometheus/common v0.59.1 // indirect + github.com/prometheus/common v0.60.1 // indirect github.com/prometheus/procfs v0.15.1 // indirect github.com/quasilyte/go-ruleguard v0.4.3-0.20240823090925-0fe6f58b47b1 // indirect github.com/quasilyte/go-ruleguard/dsl v0.3.22 // indirect @@ -274,7 +278,6 @@ require ( github.com/stbenjam/no-sprintf-host-port v0.1.1 // indirect github.com/stoewer/go-strcase v1.3.0 // indirect github.com/stretchr/objx v0.5.2 // indirect - github.com/stretchr/testify v1.10.0 // indirect github.com/subosito/gotenv v1.4.1 // indirect github.com/tdakkota/asciicheck v0.2.0 // indirect github.com/tetafro/godot v1.4.18 // indirect @@ -343,13 +346,14 @@ require ( google.golang.org/genproto/googleapis/api v0.0.0-20240903143218-8af14fe29dc1 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect google.golang.org/grpc v1.67.0 // indirect - google.golang.org/protobuf v1.34.2 // indirect + google.golang.org/protobuf v1.35.1 // indirect gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/ini.v1 v1.67.0 // indirect gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce // indirect gopkg.in/yaml.v2 v2.4.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect + gotest.tools/v3 v3.5.1 // indirect honnef.co/go/tools v0.5.1 // indirect k8s.io/apiextensions-apiserver v0.31.2 // indirect k8s.io/apiserver v0.31.2 // indirect diff --git a/go.sum b/go.sum index 82aaf77..a82da46 100644 --- a/go.sum +++ b/go.sum @@ -192,6 +192,8 @@ cloud.google.com/go/webrisk v1.4.0/go.mod h1:Hn8X6Zr+ziE2aNd8SliSDWpEnSS1u4R9+xX cloud.google.com/go/webrisk v1.5.0/go.mod h1:iPG6fr52Tv7sGk0H6qUFzmL3HHZev1htXuWDEEsqMTg= cloud.google.com/go/workflows v1.6.0/go.mod h1:6t9F5h/unJz41YqfBmqSASJSXccBLtD1Vwf+KmJENM0= cloud.google.com/go/workflows v1.7.0/go.mod h1:JhSrZuVZWuiDfKEFxU0/F1PQjmpnpcoISEXH2bcHC3M= +dario.cat/mergo v1.0.1 h1:Ra4+bf83h2ztPIQYNP99R6m+Y7KfnARDfID+a+vLl4s= +dario.cat/mergo v1.0.1/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= github.com/4meepo/tagalign v1.3.4 h1:P51VcvBnf04YkHzjfclN6BbsopfJR5rxs1n+5zHt+w8= github.com/4meepo/tagalign v1.3.4/go.mod h1:M+pnkHH2vG8+qhE5bVc/zeP7HS/j910Fwa9TUSyZVI0= @@ -397,8 +399,8 @@ github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5Xh github.com/docker/go-metrics v0.0.0-20180209012529-399ea8c73916/go.mod h1:/u0gXw0Gay3ceNrsHubL3BtdOL2fHf93USgMTe0W5dI= github.com/docker/go-metrics v0.0.1 h1:AgB/0SvBxihN0X8OR4SjsblXkbMvalQ8cjmtKQ2rQV8= github.com/docker/go-metrics v0.0.1/go.mod h1:cG1hvH2utMXtqgqqYE9plW6lDxS3/5ayHzueweSI3Vw= -github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw= -github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= +github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= +github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 h1:UhxFibDNY/bfvqU5CAUmr9zpesgbU6SWc8/B4mflAE4= github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE= github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96/go.mod h1:Qh8CwZgvJUkLughtfhJv5dyTYa91l1fOUCrgjqmcifM= @@ -945,8 +947,9 @@ github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3I github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= github.com/opencontainers/image-spec v1.0.3-0.20211202183452-c5a74bcca799 h1:rc3tiVYb5z54aKaDfakKn0dDjIyPpTtszkjuMzyt7ec= github.com/opencontainers/image-spec v1.0.3-0.20211202183452-c5a74bcca799/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= -github.com/opentracing/opentracing-go v1.1.0 h1:pWlfV3Bxv7k65HYwkikxat0+s3pV4bsqf19k25Ur8rU= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= +github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs= +github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc= github.com/otiai10/copy v1.2.0/go.mod h1:rrF5dJ5F0t/EWSYODDu4j9/vEeYHMkc8jt0zJChqQWw= github.com/otiai10/copy v1.14.0 h1:dCI/t1iTdYGtkvCuBG2BgR6KZa83PTclw4U5n2wAllU= github.com/otiai10/copy v1.14.0/go.mod h1:ECfuL02W+/FkTWZWgQqXPWZgW9oeKCSQ5qVfSc4qc4w= @@ -976,14 +979,16 @@ github.com/power-devops/perfstat v0.0.0-20220216144756-c35f1ee13d7c h1:NRoLoZvkB github.com/power-devops/perfstat v0.0.0-20220216144756-c35f1ee13d7c/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE= github.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g= github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U= +github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.78.2 h1:SyoVBXD/r0PntR1rprb90ClI32FSUNOCWqqTatnipHM= +github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.78.2/go.mod h1:SvsRXw4m1F2vk7HquU5h475bFpke27mIUswfyw9u3ug= github.com/prometheus/client_golang v0.9.0-pre1.0.20180209125602-c332b6f63c06/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v0.9.2/go.mod h1:OsXs2jCmiKlQ1lTBmv21f2mNfw4xf/QclQDMrYNZzcM= github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQP1xR9D75/vuwEF3g= -github.com/prometheus/client_golang v1.20.4 h1:Tgh3Yr67PaOv/uTqloMsCEdeuFTatm5zIq5+qNN23vI= -github.com/prometheus/client_golang v1.20.4/go.mod h1:PIEt8X02hGcP8JWbeHyeZ53Y/jReSnHgO035n//V5WE= +github.com/prometheus/client_golang v1.20.5 h1:cxppBPuYhUnsO6yo/aoRol4L7q7UFfdm+bR9r+8l63Y= +github.com/prometheus/client_golang v1.20.5/go.mod h1:PIEt8X02hGcP8JWbeHyeZ53Y/jReSnHgO035n//V5WE= github.com/prometheus/client_model v0.0.0-20171117100541-99fa1f4be8e5/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= @@ -996,8 +1001,8 @@ github.com/prometheus/common v0.0.0-20181126121408-4724e9255275/go.mod h1:daVV7q github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.6.0/go.mod h1:eBmuwkDJBwy6iBfxCBob6t6dR6ENT/y+J+Zk0j9GMYc= -github.com/prometheus/common v0.59.1 h1:LXb1quJHWm1P6wq/U824uxYi4Sg0oGvNeUm1z5dJoX0= -github.com/prometheus/common v0.59.1/go.mod h1:GpWM7dewqmVYcd7SmRaiWVe9SSqjf0UrwnYnpEZNuT0= +github.com/prometheus/common v0.60.1 h1:FUas6GcOw66yB/73KC+BOZoFJmbo/1pojoILArPAaSc= +github.com/prometheus/common v0.60.1/go.mod h1:h0LYf1R1deLSKtD4Vdg8gy4RuOvENW2J/h19V5NADQw= github.com/prometheus/procfs v0.0.0-20180125133057-cb4147076ac7/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= @@ -1883,8 +1888,8 @@ google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQ google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= -google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg= -google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw= +google.golang.org/protobuf v1.35.1 h1:m3LfL6/Ca+fqnjnlqQXNpFPABW1UD7mjh8KO2mKFytA= +google.golang.org/protobuf v1.35.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/cenkalti/backoff.v2 v2.2.1 h1:eJ9UAg01/HIHG987TwxvnzK2MgxXq97YY6rYDpY9aII= @@ -1928,8 +1933,8 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo= gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw= -gotest.tools/v3 v3.3.0 h1:MfDY1b1/0xN1CyMlQDac0ziEy9zJQd9CXBRRDHw2jJo= -gotest.tools/v3 v3.3.0/go.mod h1:Mcr9QNxkg0uMvy/YElmo4SpXgJKWgQvYrT7Kw5RzJ1A= +gotest.tools/v3 v3.5.1 h1:EENdUnS3pdur5nybKYIh2Vfgc8IUNBjxDPSjtiJcOzU= +gotest.tools/v3 v3.5.1/go.mod h1:isy3WKz7GK6uNw/sbHzfKBLvlvXwUyV06n6brMxxopU= honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 6e24ecb..4c8a5cb 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -4,6 +4,10 @@ import ( "strings" "testing" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "k8s.io/apimachinery/pkg/runtime/schema" yaml "sigs.k8s.io/yaml/goyaml.v3" ) @@ -21,33 +25,21 @@ types: cfg := Config{} err := yaml.NewDecoder(configFile).Decode(&cfg) - if err != nil { - t.Errorf("Unexpected error while decoding config: %v", err) - return - } - - if len(cfg.Types) != 2 { - t.Errorf("Unexpected length of Types: expected %v, got %v", 2, len(cfg.Types)) - return - } - - if cfg.Types[0].APIVersion != "v1" { - t.Errorf("Unexpected apiVersion: expected %v, got %v", "v1", cfg.Types[0].APIVersion) - return - } - - if cfg.Types[0].Kind != "Namespace" { - t.Errorf("Unexpected kind: expected %v, got %v", "Namespace", cfg.Types[0].Kind) - return - } - - if cfg.Types[1].APIVersion != "apps/v1" { - t.Errorf("Unexpected apiVersion: expected %v, got %v", "apps/v1", cfg.Types[1].APIVersion) - return - } - - if cfg.Types[1].Kind != "Deployment" { - t.Errorf("Unexpected Kind: expected %v, got %v", "Deployment", cfg.Types[1].Kind) - return - } + require.NoError(t, err) + + require.Len(t, cfg.Types, 2) + + assert.Equal(t, "v1", cfg.Types[0].APIVersion) + assert.Equal(t, "Namespace", cfg.Types[0].Kind) + assert.Equal(t, + schema.GroupVersionKind{Group: "", Version: "v1", Kind: "Namespace"}, + cfg.Types[0].GroupVersionKind(), + ) + + assert.Equal(t, "apps/v1", cfg.Types[1].APIVersion) + assert.Equal(t, "Deployment", cfg.Types[1].Kind) + assert.Equal(t, + schema.GroupVersionKind{Group: "apps", Version: "v1", Kind: "Deployment"}, + cfg.Types[1].GroupVersionKind(), + ) } diff --git a/internal/controller/pod_controller.go b/internal/controller/pod_controller.go new file mode 100644 index 0000000..963d87a --- /dev/null +++ b/internal/controller/pod_controller.go @@ -0,0 +1,81 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package controller + +import ( + "context" + "fmt" + + monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/klog/v2" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/log" +) + +// PodReconciler reconciles a Pod object +type PodReconciler struct { + client.Client + Scheme *runtime.Scheme + PrometheusScope *PrometheusScope +} + +// +kubebuilder:rbac:groups=core,resources=pods,verbs=get;list;watch + +// Reconcile +func (r *PodReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := log.FromContext(ctx, + "pod", klog.KRef(req.Namespace, req.Name), + ) + + ok, err := r.PrometheusScope.autoDetect.PrometheusCRsAvailability() + if err != nil { + return ctrl.Result{}, err + } else if !ok { + return ctrl.Result{}, ErrPrometheusCRsNotAvailable + } + + log.V(2).Info("Reconciling") + + pod := &corev1.Pod{} + if err := r.Get(ctx, req.NamespacedName, pod); err != nil { + return ctrl.Result{}, fmt.Errorf("unable to get pod: %w", err) + } + + // TODO: check if has monitoring enabled + // TODO: get all annotation + + isMarkedToBeDeleted := pod.GetDeletionTimestamp() != nil + if isMarkedToBeDeleted { + // TODO: handle deletion of the adjacent resources + } + + // TODO: handle creation/update of the resources + + return ctrl.Result{}, nil +} + +// SetupWithManager sets up the controller with the Manager. +func (r *PodReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&corev1.Pod{}). + Owns(&monitoringv1.PodMonitor{}). + Complete(r) +} diff --git a/internal/controller/pod_controller_test.go b/internal/controller/pod_controller_test.go new file mode 100644 index 0000000..b53f324 --- /dev/null +++ b/internal/controller/pod_controller_test.go @@ -0,0 +1,17 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package controller diff --git a/internal/controller/prometheusscope.go b/internal/controller/prometheusscope.go new file mode 100644 index 0000000..d66a923 --- /dev/null +++ b/internal/controller/prometheusscope.go @@ -0,0 +1,97 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package controller + +import ( + "errors" + + "k8s.io/client-go/discovery" + "k8s.io/client-go/rest" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// +kubebuilder:rbac:groups=monitoring.coreos.com,resources=servicemonitors,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=monitoring.coreos.com,resources=podmonitors,verbs=get;list;watch;create;update;patch;delete + +var ErrPrometheusCRsNotAvailable = errors.New("prometheus custom resources cannot be found on cluster") + +const ( + prometheusScrapeAnnotation = "prometheus.io/scrape" + prometheusPortAnnotation = "prometheus.io/port" + prometheusPathAnnotation = "prometheus.io/path" + + monitorsAnnotation = "scribe.anza-labs.dev/monitors" + monitorsAnnotationDisabled = "disabled" +) + +type PrometheusScope struct { + client.Client + autoDetect autoDetect +} + +func NewPrometheusScope(c client.Client, cfg *rest.Config) (*PrometheusScope, error) { + dcl, err := discovery.NewDiscoveryClientForConfig(cfg) + if err != nil { + return nil, err + } + + return &PrometheusScope{ + Client: c, + autoDetect: autoDetect{ + DiscoveryInterface: dcl, + }, + }, nil +} + +type autoDetect struct { + discovery.DiscoveryInterface +} + +func (a *autoDetect) PrometheusCRsAvailability() (bool, error) { + apiList, err := a.ServerGroups() + if err != nil { + return false, err + } + + foundServiceMonitor := false + foundPodMonitor := false + apiGroups := apiList.Groups + for i := 0; i < len(apiGroups); i++ { + if apiGroups[i].Name == "monitoring.coreos.com" { + for _, version := range apiGroups[i].Versions { + resources, err := a.ServerResourcesForGroupVersion(version.GroupVersion) + if err != nil { + return false, err + } + + for _, resource := range resources.APIResources { + if resource.Kind == "ServiceMonitor" { + foundServiceMonitor = true + } else if resource.Kind == "PodMonitor" { + foundPodMonitor = true + } + } + } + } + } + + if foundServiceMonitor && foundPodMonitor { + return true, nil + } + + return false, nil +} diff --git a/internal/controller/prometheusscope_test.go b/internal/controller/prometheusscope_test.go new file mode 100644 index 0000000..b53f324 --- /dev/null +++ b/internal/controller/prometheusscope_test.go @@ -0,0 +1,17 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package controller diff --git a/internal/controller/service_controller.go b/internal/controller/service_controller.go new file mode 100644 index 0000000..d1c8ef8 --- /dev/null +++ b/internal/controller/service_controller.go @@ -0,0 +1,81 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package controller + +import ( + "context" + "fmt" + + monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/klog/v2" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/log" +) + +// ServiceReconciler reconciles a Service object +type ServiceReconciler struct { + client.Client + Scheme *runtime.Scheme + PrometheusScope *PrometheusScope +} + +// +kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch + +// Reconcile +func (r *ServiceReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := log.FromContext(ctx, + "service", klog.KRef(req.Namespace, req.Name), + ) + + ok, err := r.PrometheusScope.autoDetect.PrometheusCRsAvailability() + if err != nil { + return ctrl.Result{}, err + } else if !ok { + return ctrl.Result{}, ErrPrometheusCRsNotAvailable + } + + log.V(2).Info("Reconciling") + + svc := &corev1.Service{} + if err := r.Get(ctx, req.NamespacedName, svc); err != nil { + return ctrl.Result{}, fmt.Errorf("unable to get pod: %w", err) + } + + // TODO: check if has monitoring enabled + // TODO: get all annotation + + isMarkedToBeDeleted := svc.GetDeletionTimestamp() != nil + if isMarkedToBeDeleted { + // TODO: handle deletion of the adjacent resources + } + + // TODO: handle creation/update of the resources + + return ctrl.Result{}, nil +} + +// SetupWithManager sets up the controller with the Manager. +func (r *ServiceReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&corev1.Service{}). + Owns(&monitoringv1.ServiceMonitor{}). + Complete(r) +} diff --git a/internal/controller/service_controller_test.go b/internal/controller/service_controller_test.go new file mode 100644 index 0000000..b53f324 --- /dev/null +++ b/internal/controller/service_controller_test.go @@ -0,0 +1,17 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package controller diff --git a/internal/manifests/builder.go b/internal/manifests/builder.go new file mode 100644 index 0000000..59591de --- /dev/null +++ b/internal/manifests/builder.go @@ -0,0 +1,44 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The OpenTelemetry Authors + +package manifests + +import ( + "reflect" + + "sigs.k8s.io/controller-runtime/pkg/client" +) + +type Builder[Params any] func(params Params) ([]client.Object, error) + +type ManifestFactory[T client.Object, Params any] func(params Params) (T, error) +type K8sManifestFactory[Params any] ManifestFactory[client.Object, Params] + +func Factory[T client.Object, Params any](f ManifestFactory[T, Params]) K8sManifestFactory[Params] { + return func(params Params) (client.Object, error) { + return f(params) + } +} + +// ObjectIsNotNil ensures that we only create an object IFF it isn't nil, +// and it's concrete type isn't nil either. This works around the Go type system +// by using reflection to verify its concrete type isn't nil. +func ObjectIsNotNil(obj client.Object) bool { + return obj != nil && !reflect.ValueOf(obj).IsNil() +} diff --git a/internal/manifests/manifestutils/annotations.go b/internal/manifests/manifestutils/annotations.go new file mode 100644 index 0000000..8cc0484 --- /dev/null +++ b/internal/manifests/manifestutils/annotations.go @@ -0,0 +1,38 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The OpenTelemetry Authors + +package manifestutils + +import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + +// Annotations return the annotations for the resources. +func Annotations(instance metav1.ObjectMeta, filterAnnotations []string) (map[string]string, error) { + // new map every time, so that we don't touch the instance's annotations + annotations := map[string]string{} + + if nil != instance.Annotations { + for k, v := range instance.Annotations { + if !IsFilteredSet(k, filterAnnotations) { + annotations[k] = v + } + } + } + + return annotations, nil +} diff --git a/internal/manifests/manifestutils/annotations_test.go b/internal/manifests/manifestutils/annotations_test.go new file mode 100644 index 0000000..fcdefcd --- /dev/null +++ b/internal/manifests/manifestutils/annotations_test.go @@ -0,0 +1,60 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package manifestutils + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func TestAnnotationsPropagateDown(t *testing.T) { + // prepare + meta := metav1.ObjectMeta{ + Annotations: map[string]string{"myapp": "mycomponent"}, + } + + // test + annotations, err := Annotations(meta, []string{}) + require.NoError(t, err) + + // verify + assert.Len(t, annotations, 1) + assert.Equal(t, "mycomponent", annotations["myapp"]) +} + +func TestAnnotationsFilter(t *testing.T) { + meta := metav1.ObjectMeta{ + Annotations: map[string]string{ + "test.bar.io": "foo", + "test.io/port": "1234", + "test.io/path": "/test", + }, + } + + // This requires the filter to be in regex match form and not the other simpler wildcard one. + annotations, err := Annotations(meta, []string{".*\\.bar\\.io"}) + + // verify + require.NoError(t, err) + assert.Len(t, annotations, 2) + assert.NotContains(t, annotations, "test.bar.io") + assert.Equal(t, "1234", annotations["test.io/port"]) +} diff --git a/internal/manifests/manifestutils/labels.go b/internal/manifests/manifestutils/labels.go new file mode 100644 index 0000000..0def768 --- /dev/null +++ b/internal/manifests/manifestutils/labels.go @@ -0,0 +1,78 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The OpenTelemetry Authors + +package manifestutils + +import ( + "regexp" + + "github.com/anza-labs/scribe/internal/naming" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func IsFilteredSet(sourceSet string, filterSet []string) bool { + for _, basePattern := range filterSet { + pattern, _ := regexp.Compile(basePattern) + if match := pattern.MatchString(sourceSet); match { + return match + } + } + return false +} + +// Labels return the common labels to all objects that are part of a managed CR. +func Labels( + instance metav1.ObjectMeta, + name string, + component string, + filterLabels []string, +) map[string]string { + // new map every time, so that we don't touch the instance's label + base := map[string]string{} + if nil != instance.Labels { + for k, v := range instance.Labels { + if !IsFilteredSet(k, filterLabels) { + base[k] = v + } + } + } + + for k, v := range SelectorLabels(instance, component) { + base[k] = v + } + + // Don't override the app name if it already exists + if _, ok := base["app.kubernetes.io/name"]; !ok { + base["app.kubernetes.io/name"] = name + } + return base +} + +// SelectorLabels return the common labels to all objects that are part of a managed CR to use as selector. +// Selector labels are immutable for Deployment, StatefulSet and DaemonSet, therefore, no labels in selector should be +// expected to be modified for the lifetime of the object. +func SelectorLabels(instance metav1.ObjectMeta, component string) map[string]string { + return map[string]string{ + "app.kubernetes.io/managed-by": "scribe", + "app.kubernetes.io/instance": naming.Truncate("%s.%s", 63, instance.Namespace, instance.Name), + "app.kubernetes.io/part-of": "monitoring", + "app.kubernetes.io/component": component, + } +} diff --git a/internal/manifests/manifestutils/labels_test.go b/internal/manifests/manifestutils/labels_test.go new file mode 100644 index 0000000..8097c9f --- /dev/null +++ b/internal/manifests/manifestutils/labels_test.go @@ -0,0 +1,80 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The registry Authors + +package manifestutils + +import ( + "testing" + + "github.com/stretchr/testify/assert" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +const ( + objectName = "my-instance" + objectNamespace = "my-ns" +) + +func TestLabelsCommonSet(t *testing.T) { + // prepare + meta := metav1.ObjectMeta{ + Name: objectName, + Namespace: objectNamespace, + } + + // test + labels := Labels(meta, objectName, "podmonitor", []string{}) + assert.Equal(t, "scribe", labels["app.kubernetes.io/managed-by"]) + assert.Equal(t, "my-ns.my-instance", labels["app.kubernetes.io/instance"]) + assert.Equal(t, "monitoring", labels["app.kubernetes.io/part-of"]) + assert.Equal(t, "registry", labels["app.kubernetes.io/component"]) +} + +func TestLabelsPropagateDown(t *testing.T) { + // prepare + meta := metav1.ObjectMeta{ + Labels: map[string]string{ + "myapp": "mycomponent", + "app.kubernetes.io/name": "test", + }, + } + + // test + labels := Labels(meta, objectName, "podmonitor", []string{}) + + // verify + assert.Len(t, labels, 7) + assert.Equal(t, "mycomponent", labels["myapp"]) + assert.Equal(t, "test", labels["app.kubernetes.io/name"]) +} + +func TestLabelsFilter(t *testing.T) { + meta := metav1.ObjectMeta{ + Labels: map[string]string{"test.bar.io": "foo", "test.foo.io": "bar"}, + } + + // This requires the filter to be in regex match form and not the other simpler wildcard one. + labels := Labels(meta, objectName, "registry", []string{".*.bar.io"}) + + // verify + assert.Len(t, labels, 7) + assert.NotContains(t, labels, "test.bar.io") + assert.Equal(t, "bar", labels["test.foo.io"]) +} diff --git a/internal/manifests/mutate.go b/internal/manifests/mutate.go new file mode 100644 index 0000000..391ba03 --- /dev/null +++ b/internal/manifests/mutate.go @@ -0,0 +1,100 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The OpenTelemetry Authors + +package manifests + +import ( + "errors" + "fmt" + "reflect" + + "dario.cat/mergo" + monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" +) + +var ( + ErrImmutableChange = errors.New("immutable field change attempted") +) + +// MutateFuncFor returns a mutate function based on the +// existing resource's concrete type. It supports currently +// only the following types or else panics: +// - PodMonitor +// - ServiceMonitor +// In order for the operator to reconcile other types, they must be added here. +// The function returned takes no arguments but instead uses the existing and desired inputs here. Existing is expected +// to be set by the controller-runtime package through a client get call. +func MutateFuncFor(existing, desired client.Object) controllerutil.MutateFn { + return func() error { + // Get the existing annotations and override any conflicts with the desired annotations + // This will preserve any annotations on the existing set. + existingAnnotations := existing.GetAnnotations() + if err := mergeWithOverride(&existingAnnotations, desired.GetAnnotations()); err != nil { + return err + } + existing.SetAnnotations(existingAnnotations) + + // Get the existing labels and override any conflicts with the desired labels + // This will preserve any labels on the existing set. + existingLabels := existing.GetLabels() + if err := mergeWithOverride(&existingLabels, desired.GetLabels()); err != nil { + return err + } + existing.SetLabels(existingLabels) + + if ownerRefs := desired.GetOwnerReferences(); len(ownerRefs) > 0 { + existing.SetOwnerReferences(ownerRefs) + } + + switch existing.(type) { + case *monitoringv1.PodMonitor: + pm := existing.(*monitoringv1.PodMonitor) + wantPm := desired.(*monitoringv1.PodMonitor) + mutatePodMonitor(pm, wantPm) + + case *monitoringv1.ServiceMonitor: + sm := existing.(*monitoringv1.ServiceMonitor) + wantSm := desired.(*monitoringv1.ServiceMonitor) + mutateServiceMonitor(sm, wantSm) + + default: + t := reflect.TypeOf(existing).String() + return fmt.Errorf("missing mutate implementation for resource type: %s", t) + } + return nil + } +} + +func mergeWithOverride(dst, src interface{}) error { + return mergo.Merge(dst, src, mergo.WithOverride) +} + +func mutatePodMonitor(existing, desired *monitoringv1.PodMonitor) { + existing.Spec.PodMetricsEndpoints = desired.Spec.PodMetricsEndpoints + existing.Spec.Selector = desired.Spec.Selector +} + +func mutateServiceMonitor(existing, desired *monitoringv1.ServiceMonitor) { + // TODO: write mutation + existing.Spec.Endpoints = desired.Spec.Endpoints + existing.Spec.Selector = desired.Spec.Selector +} diff --git a/internal/manifests/prometheus/podmonitor.go b/internal/manifests/prometheus/podmonitor.go new file mode 100644 index 0000000..3fcccc8 --- /dev/null +++ b/internal/manifests/prometheus/podmonitor.go @@ -0,0 +1,60 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package prometheus + +import ( + monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + + "github.com/anza-labs/scribe/internal/manifests/manifestutils" + "github.com/anza-labs/scribe/internal/naming" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func PodMonitor(pod *corev1.Pod) (*monitoringv1.PodMonitor, error) { + name := naming.PodMonitor(pod.Name) + labels := manifestutils.Labels(pod.ObjectMeta, name, ComponentPodMonitor, nil) + annotations, err := manifestutils.Annotations(pod.ObjectMeta, nil) + if err != nil { + return nil, err + } + + if len(pod.Labels) == 0 { + return nil, ErrMissingLabels + } + selector := metav1.SetAsLabelSelector(pod.Labels) + + return &monitoringv1.PodMonitor{ + ObjectMeta: metav1.ObjectMeta{ + Name: name, + Namespace: pod.Namespace, + Labels: labels, + Annotations: annotations, + }, + Spec: monitoringv1.PodMonitorSpec{ + + Selector: *selector, + NamespaceSelector: monitoringv1.NamespaceSelector{ + MatchNames: []string{ + pod.Namespace, + }, + }, + PodMetricsEndpoints: []monitoringv1.PodMetricsEndpoint{}, // TODO: extract this to function + }, + }, nil +} diff --git a/internal/manifests/prometheus/prometheus.go b/internal/manifests/prometheus/prometheus.go new file mode 100644 index 0000000..bf8e61e --- /dev/null +++ b/internal/manifests/prometheus/prometheus.go @@ -0,0 +1,28 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package prometheus + +import "errors" + +const ( + ComponentPodMonitor = "podmonitor" + ComponentServiceMonitor = "servicemonitor" +) + +var ( + ErrMissingLabels = errors.New("no labels defined on object, label selector can't be constructed") +) diff --git a/internal/manifests/prometheus/servicemonitor.go b/internal/manifests/prometheus/servicemonitor.go new file mode 100644 index 0000000..661af25 --- /dev/null +++ b/internal/manifests/prometheus/servicemonitor.go @@ -0,0 +1,59 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package prometheus + +import ( + monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + + "github.com/anza-labs/scribe/internal/manifests/manifestutils" + "github.com/anza-labs/scribe/internal/naming" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func ServiceMonitor(svc *corev1.Service) (*monitoringv1.ServiceMonitor, error) { + name := naming.ServiceMonitor(svc.Name) + labels := manifestutils.Labels(svc.ObjectMeta, name, ComponentServiceMonitor, nil) + annotations, err := manifestutils.Annotations(svc.ObjectMeta, nil) + if err != nil { + return nil, err + } + + if len(svc.Labels) == 0 { + return nil, ErrMissingLabels + } + selector := metav1.SetAsLabelSelector(svc.Labels) + + return &monitoringv1.ServiceMonitor{ + ObjectMeta: metav1.ObjectMeta{ + Name: name, + Namespace: svc.Namespace, + Labels: labels, + Annotations: annotations, + }, + Spec: monitoringv1.ServiceMonitorSpec{ + Selector: *selector, + NamespaceSelector: monitoringv1.NamespaceSelector{ + MatchNames: []string{ + svc.Namespace, + }, + }, + Endpoints: []monitoringv1.Endpoint{}, // TODO: extract this to function + }, + }, nil +} diff --git a/internal/naming/dns.go b/internal/naming/dns.go new file mode 100644 index 0000000..53f7646 --- /dev/null +++ b/internal/naming/dns.go @@ -0,0 +1,51 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The Jaeger Authors + +package naming + +import ( + "regexp" + "strings" + "unicode/utf8" +) + +var regex = regexp.MustCompile(`[a-z0-9]`) + +// DNSName returns a dns-safe string for the given name. +// Any char that is not [a-z0-9] is replaced by "-" or "a". +// Replacement character "a" is used only at the beginning or at the end of the name. +// The function does not change length of the string. +// source: https://github.com/jaegertracing/jaeger-operator/blob/91e3b69ee5c8761bbda9d3cf431400a73fc1112a/pkg/util/dns_name.go#L15 +func DNSName(name string) string { + var d []rune + + for i, x := range strings.ToLower(name) { + if regex.Match([]byte(string(x))) { + d = append(d, x) + } else { + if i == 0 || i == utf8.RuneCountInString(name)-1 { + d = append(d, 'a') + } else { + d = append(d, '-') + } + } + } + + return string(d) +} diff --git a/internal/naming/dns_test.go b/internal/naming/dns_test.go new file mode 100644 index 0000000..c747d2b --- /dev/null +++ b/internal/naming/dns_test.go @@ -0,0 +1,52 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The Jaeger Authors + +package naming + +import ( + "regexp" + "testing" + + "github.com/stretchr/testify/assert" +) + +func TestDnsName(t *testing.T) { + var tests = []struct { + in string + out string + }{ + {"simplest", "simplest"}, + {"instance.with.dots-object-headless", "instance-with-dots-object-headless"}, + {"TestQueryDottedServiceName.With.Dots", "testquerydottedservicename-with-dots"}, + {"Service🦄", "servicea"}, + {"📈Stock-Tracker", "astock-tracker"}, + {"-📈Stock-Tracker", "a-stock-tracker"}, + {"📈", "a"}, + {"foo-", "fooa"}, + {"-foo", "afoo"}, + } + rule, err := regexp.Compile(`^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`) + assert.NoError(t, err) + + for _, tt := range tests { + assert.Equal(t, tt.out, DNSName(tt.in)) + matched := rule.Match([]byte(tt.out)) + assert.True(t, matched, "%v is not a valid name", tt.out) + } +} diff --git a/internal/naming/naming.go b/internal/naming/naming.go new file mode 100644 index 0000000..9328bae --- /dev/null +++ b/internal/naming/naming.go @@ -0,0 +1,28 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package naming is for determining the names for components (containers, services, ...). +package naming + +// ServiceMonitor builds the ServiceMonitor name based on the instance. +func ServiceMonitor(instance string) string { + return DNSName(Truncate("%s-sm", 63, instance)) +} + +// PodMonitor builds the PodMonitor name based on the instance. +func PodMonitor(instance string) string { + return DNSName(Truncate("%s-pm", 63, instance)) +} diff --git a/internal/naming/trimming.go b/internal/naming/trimming.go new file mode 100644 index 0000000..648f6aa --- /dev/null +++ b/internal/naming/trimming.go @@ -0,0 +1,76 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The Jaeger Authors + +package naming + +import ( + "fmt" + "regexp" +) + +var regexpEndReplace, regexpBeginReplace *regexp.Regexp + +func init() { + regexpEndReplace, _ = regexp.Compile("[^A-Za-z0-9]+$") + regexpBeginReplace, _ = regexp.Compile("^[^A-Za-z0-9]+") +} + +// Truncate will shorten the length of the instance name so that it contains at most max chars when combined with the +// fixed part. If the fixed part is already bigger than the max, this function is noop. +// source: https://github.com/jaegertracing/jaeger-operator/blob/91e3b69ee5c8761bbda9d3cf431400a73fc1112a/pkg/util/truncate.go#L17 +func Truncate(format string, max int, values ...interface{}) string { + var truncated []interface{} + result := fmt.Sprintf(format, values...) + if excess := len(result) - max; excess > 0 { + // we try to reduce the first string we find + for _, value := range values { + if excess == 0 { + truncated = append(truncated, value) + continue + } + + if s, ok := value.(string); ok { + if len(s) > excess { + value = s[:len(s)-excess] + excess = 0 + } else { + value = "" // skip this value entirely + excess = excess - len(s) + } + } + + truncated = append(truncated, value) + } + result = fmt.Sprintf(format, truncated...) + } + + // if at this point, the result is still bigger than max, apply a hard cap: + if len(result) > max { + return result[:max] + } + + return trimNonAlphaNumeric(result) +} + +// trimNonAlphaNumeric remove all non-alphanumeric values from start and end of the string +// source: https://github.com/jaegertracing/jaeger-operator/blob/91e3b69ee5c8761bbda9d3cf431400a73fc1112a/pkg/util/truncate.go#L53 +func trimNonAlphaNumeric(text string) string { + newText := regexpEndReplace.ReplaceAllString(text, "") + return regexpBeginReplace.ReplaceAllString(newText, "") +} diff --git a/internal/naming/trimming_test.go b/internal/naming/trimming_test.go new file mode 100644 index 0000000..0c63075 --- /dev/null +++ b/internal/naming/trimming_test.go @@ -0,0 +1,115 @@ +/* +Copyright 2024 anza-labs contributors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Additional copyrights: +// Copyright The Jaeger Authors + +package naming + +import ( + "testing" + + "github.com/stretchr/testify/assert" +) + +func TestTruncate(t *testing.T) { + for _, tt := range []struct { + format string + expected string + cap string + values []interface{} + max int + }{ + { + format: "%s-object", + max: 63, + values: []interface{}{"simplest"}, + expected: "simplest-object", + cap: "the standard case", + }, + { + format: "d0c1e62-4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11ea-b174-c85b7644b6b5", + max: 63, + values: []interface{}{}, + expected: "d0c1e62-4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11ea-b174-c85", + cap: "first N case", + }, + { + format: "%s-object", + max: 63, + values: []interface{}{"d0c1e62-4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11ea-b174-c85b7644b6b5"}, + expected: "d0c1e62-4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11e-object", + cap: "instance + fixed within bounds", + }, + { + format: "%s-%s-object", + max: 63, + values: []interface{}{"d0c1e62", "4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11ea-b174-c85b7644b6b5"}, + expected: "4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11ea-b174--object", + cap: "first value gets dropped, second truncated", + }, + { + format: "%s-%s-object", + max: 63, + values: []interface{}{"4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11ea-b174-c85b7644b6b5", "d0c1e62"}, + expected: "4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11e-d0c1e62-object", + cap: "first value gets truncated, second added", + }, + { + format: "%d-%s-object", + max: 63, + values: []interface{}{42, "d0c1e62-4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96-11ea-b174-c85b7644b6b5"}, + expected: "42-d0c1e62-4d96-11ea-b174-c85b7644b6b5-5d0c1e62-4d96--object", + cap: "first value gets passed, second truncated", + }, + } { + t.Run(tt.cap, func(t *testing.T) { + assert.Equal(t, tt.expected, Truncate(tt.format, tt.max, tt.values...)) + }) + } +} + +func TestTrimNonAlphaNumeric(t *testing.T) { + tests := []struct { + input string + expected string + }{ + { + input: "-%$#ThisIsALabel", + expected: "ThisIsALabel", + }, + + { + input: "label-invalid--_truncated-.", + expected: "label-invalid--_truncated", + }, + + { + input: "--(label-invalid--_truncated-#.1.", + expected: "label-invalid--_truncated-#.1", + }, + + { + input: "12ValidLabel3", + expected: "12ValidLabel3", + }, + } + + for _, test := range tests { + output := trimNonAlphaNumeric(test.input) + assert.Equal(t, test.expected, output) + } +} diff --git a/test/e2e/chainsaw-test.yaml b/test/e2e/annotations/chainsaw-test.yaml similarity index 83% rename from test/e2e/chainsaw-test.yaml rename to test/e2e/annotations/chainsaw-test.yaml index b8bc5b8..9b2bf8b 100644 --- a/test/e2e/chainsaw-test.yaml +++ b/test/e2e/annotations/chainsaw-test.yaml @@ -2,10 +2,10 @@ apiVersion: chainsaw.kyverno.io/v1alpha1 kind: Test metadata: - name: quick-start + name: annotations spec: steps: - - name: "foo" + - name: Check if Scribe is installed try: - assert: resource: @@ -16,6 +16,8 @@ spec: namespace: scribe-system status: availableReplicas: 1 + - name: Create annotated Namespace and test Deployment + try: - apply: resource: apiVersion: v1 @@ -41,7 +43,9 @@ spec: spec: containers: - name: test - image: nginx + image: docker.io/library/nginx:latest + - name: Validate if the annotation is propagated + try: - assert: resource: apiVersion: apps/v1 diff --git a/test/e2e/monitoring/podmonitor/chainsaw-test.yaml b/test/e2e/monitoring/podmonitor/chainsaw-test.yaml new file mode 100644 index 0000000..31bff7b --- /dev/null +++ b/test/e2e/monitoring/podmonitor/chainsaw-test.yaml @@ -0,0 +1,127 @@ +# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: podmonitors +spec: + steps: + - name: Check if Scribe is installed + try: + - assert: + resource: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: scribe-controller-manager + namespace: scribe-system + status: + availableReplicas: 1 + + - name: Create Pod with scrape annotation + try: + - create: + resource: + apiVersion: v1 + kind: Pod + metadata: + name: test-pod-default + annotations: + prometheus.io/scrape: 'true' + spec: + containers: + - name: test + image: docker.io/library/nginx:latest + - name: Validate if a PodMonitor from Pod with scrape annotation is created + try: + - assert: + resource: + + - name: Create Pod with scrape and path annotations + try: + - create: + resource: + apiVersion: v1 + kind: Pod + metadata: + name: test-pod-path + annotations: + prometheus.io/scrape: 'true' + prometheus.io/path: '/foo/metrics' + spec: + containers: + - name: test + image: docker.io/library/nginx:latest + - name: Validate if a PodMonitor from Pod with scrape and path annotations is created + try: + - assert: + resource: + + - name: Create Pod with scrape and port annotations + try: + - create: + resource: + apiVersion: v1 + kind: Pod + metadata: + name: test-pod-port + annotations: + prometheus.io/scrape: 'true' + prometheus.io/port: '9023' + spec: + containers: + - name: test + image: docker.io/library/nginx:latest + - name: Validate if a PodMonitor from Pod with scrape and port annotations is created + try: + - assert: + resource: + + - name: Create Pod with all annotations + try: + - create: + resource: + apiVersion: v1 + kind: Pod + metadata: + name: test-pod-full + annotations: + prometheus.io/scrape: 'true' + prometheus.io/path: '/foo/metrics' + prometheus.io/port: '9023' + spec: + containers: + - name: test + image: docker.io/library/nginx:latest + - name: Validate if a PodMonitor from Pod with all annotations is created + try: + - assert: + resource: + + - name: Create Deployment with all annotations + try: + - create: + resource: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: test-deployment-full + spec: + selector: + matchLabels: + app: test + template: + metadata: + labels: + app: test + annotations: + prometheus.io/scrape: 'true' + prometheus.io/path: '/foo/metrics' + prometheus.io/port: '9023' + spec: + containers: + - name: test + image: docker.io/library/nginx:latest + - name: Validate if a single PodMonitor from Pods created by Deployment with all annotations is created + try: + - assert: + resource: diff --git a/test/e2e/monitoring/servicemonitor/chainsaw-test.yaml b/test/e2e/monitoring/servicemonitor/chainsaw-test.yaml new file mode 100644 index 0000000..8e74ceb --- /dev/null +++ b/test/e2e/monitoring/servicemonitor/chainsaw-test.yaml @@ -0,0 +1,170 @@ +# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: podmonitors +spec: + steps: + - name: Check if Scribe is installed + try: + - assert: + resource: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: scribe-controller-manager + namespace: scribe-system + status: + availableReplicas: 1 + + - name: Create Service with scrape annotation + try: + - create: + resource: + apiVersion: v1 + kind: Service + metadata: + name: test-svc-default + annotations: + prometheus.io/scrape: 'true' + labels: + monitor: test + spec: + selector: + app: test + ports: + - port: web + targetPort: 8080 + - name: Validate if a ServiceMonitor from Service with scrape annotation is created + try: + - assert: + resource: + apiVersion: monitoring.coreos.com/v1 + kind: ServiceMonitor + metadata: + name: test-svc-default + labels: + monitor: test + spec: + selector: + matchLabels: + app: test + endpoints: + - port: web + targetPort: 8080 + path: '/metrics' + + - name: Create Service with scrape and path annotations + try: + - create: + resource: + apiVersion: v1 + kind: Service + metadata: + name: test-svc-path + annotations: + prometheus.io/scrape: 'true' + prometheus.io/path: '/foo/metrics' + labels: + monitor: test + spec: + selector: + app: test + ports: + - port: web + targetPort: 8080 + - name: Validate if a ServiceMonitor from Service with scrape and path annotations is created + try: + - assert: + resource: + apiVersion: monitoring.coreos.com/v1 + kind: ServiceMonitor + metadata: + name: test-svc-path + labels: + monitor: test + spec: + selector: + matchLabels: + app: test + endpoints: + - port: web + targetPort: 8080 + path: '/foo/metrics' + + - name: Create Service with scrape and port annotations + try: + - create: + resource: + apiVersion: v1 + kind: Service + metadata: + name: test-svc-port + annotations: + prometheus.io/scrape: 'true' + prometheus.io/port: 'custom' + spec: + selector: + app: test + ports: + - port: web + targetPort: 8080 + - port: custom + targetPort: 9023 + - name: Validate if a ServiceMonitor from Service with scrape and port annotations is created + try: + - assert: + resource: + apiVersion: monitoring.coreos.com/v1 + kind: ServiceMonitor + metadata: + name: test-svc-port + labels: + monitor: test + spec: + selector: + matchLabels: + app: test + endpoints: + - port: custom + targetPort: 9023 + path: '/foo/metrics' + + - name: Create Service with all annotations + try: + - create: + resource: + apiVersion: v1 + kind: Service + metadata: + name: test-svc-full + annotations: + prometheus.io/scrape: 'true' + prometheus.io/path: '/foo/metrics' + prometheus.io/port: 'custom' + spec: + selector: + app: test + ports: + - port: web + targetPort: 8080 + - port: custom + targetPort: 9023 + - name: Validate if a ServiceMonitor from Service with all annotations is created + try: + - assert: + resource: + apiVersion: monitoring.coreos.com/v1 + kind: ServiceMonitor + metadata: + name: test-svc-all + labels: + monitor: test + spec: + selector: + matchLabels: + app: test + endpoints: + - port: custom + targetPort: 9023 + path: '/foo/metrics'