Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bitnami/kube-prometheus]: support tsdb runtime configuration #25444

Closed
rfc2119 opened this issue Apr 29, 2024 · 8 comments
Closed

[bitnami/kube-prometheus]: support tsdb runtime configuration #25444

rfc2119 opened this issue Apr 29, 2024 · 8 comments
Assignees
Labels
feature-request kube-prometheus solved stale 15 days without activity triage Triage is needed

Comments

@rfc2119
Copy link

rfc2119 commented Apr 29, 2024

Name and Version

bitnami/kube-prometheus 9.0.5

What is the problem this feature will solve?

The TSDB enables support for configuring the time window out-of-order samples. It was implemented here prometheus/prometheus#11075 in Prometheus v2.39.0, and the Prometheus operator supports configuring it. However, the generated prometheus.env.yaml file does not include that option, and also the documentation for the helm chart does not mention it. Here's an example "values.yaml" file:

prometheus:
  retention: 30d
  tsdb:
    outOfOrderTimeWindow: 5m

And in the Prometheus config file itself: Configuration | Prometheus

What is the feature you are proposing to solve the problem?

Support configuring TSDB's runtime options, including out-of-order samples. The latter option is still considered experimental, but it's widely used in production and is effective in large clusters.

@carrodher
Copy link
Member

Thank you for bringing this issue to our attention. We appreciate your involvement! If you're interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

@rfc2119
Copy link
Author

rfc2119 commented May 12, 2024

Thanks @carrodher . Before submitting a PR, I was trying to test this out on a live version bitnami/kube-prometheus chart (9.0.7). I have edited the CRD "monitoring.coreos.com/v1/prometheus" on-the-fly to include the relevant stanza, like this:

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  annotations:
    meta.helm.sh/release-name: prometheus
  labels:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/instance: prometheus
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kube-prometheus
    app.kubernetes.io/version: 0.73.2
    helm.sh/chart: kube-prometheus-9.0.7
spec:
  additionalScrapeConfigs:
    key: scrape-jobs.yaml
    name: additional-scrape-jobs-prometheus-prometheus
  tsdb:    # <-------------------
    outOfOrderTimeWindow: 2m

However, I am getting

strict decoding error: unknown field "spec.tsdb"

The operator clearly supports TSDBSpec inside PrometheusSpec. I have no mutating or validating webhooks. Any idea why this errors ?

@javsalgar
Copy link
Contributor

Is the operator throwing that error or the k8s api?

@rfc2119
Copy link
Author

rfc2119 commented May 13, 2024

k8s API, while submitting the YAML file.

@javsalgar
Copy link
Contributor

Does the issue happen on a fresh installation where you deploy the YAML directly modified instead of patching an already existing instance?

@rfc2119
Copy link
Author

rfc2119 commented May 14, 2024

I upgraded an existing installation from 8.x.x to 9.0.7, then tried to patch the manifest file online.
EDIT: kubectl describe crd prometheuses.monitoring.coreos.com does not contain tsdb field. Is this normal ?
EDIT 2: I reinstalled the CRDs manually from hack/update_crds.sh and I can test it successfully. No idea why it didn't run while updating the helm chart

Copy link

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

@github-actions github-actions bot added the stale 15 days without activity label May 30, 2024
Copy link

github-actions bot commented Jun 4, 2024

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

@github-actions github-actions bot added the solved label Jun 4, 2024
@bitnami-bot bitnami-bot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request kube-prometheus solved stale 15 days without activity triage Triage is needed
Projects
None yet
Development

No branches or pull requests

4 participants