Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support explicit endpoint for Prometheus exporter. #6071

Closed
nluk opened this issue Dec 11, 2023 · 7 comments
Closed

Support explicit endpoint for Prometheus exporter. #6071

nluk opened this issue Dec 11, 2023 · 7 comments

Comments

@nluk
Copy link

nluk commented Dec 11, 2023

Is your feature request related to a problem? Please describe.

Long story short: Prometheus exporter responds to GET requests on any path/endpoint:

  • /metrics
  • /liveness
  • /my/pets/123
  • /

In my case, I didn't notice a missing update in a k8s manifest - custom listener on port 12321 used to handle both metrics and liveness/readiness. I've switched liveness to the application port, and kept 12321 for Prometheus exporter. I forgot to update the probes, and the exporter responded with 200 OK to probe calls on /liveness and /readiness, thus hiding the missconfiguration.

Describe the solution you'd like

New enviromental variable:
OTEL_EXPORTER_PROMETHEUS_PATH=/metrics

or:
OTEL_EXPORTER_PROMETHEUS_PATHS=/metrics,/foo,/

Also, 404 response for requests that do not match the endpoint.

Describe alternatives you've considered

No response

Additional context

No response

@jack-berg
Copy link
Member

Moving to opentelemetry-java, which is where there prometheus exporter is defined.

@jack-berg jack-berg transferred this issue from open-telemetry/opentelemetry-java-instrumentation Dec 12, 2023
@fstab
Copy link
Member

fstab commented Dec 21, 2023

I'm happy to implement this after #6015 is merged, it should be straightforward.

I'm wondering: The Prometheus client_java library had the same issue, and with the 1.x release we changed the default behavior to only export metrics on /metrics.

In the OTel SDK we left the previous behavior, exporting metrics on all HTTP paths. However, we could change the default as well and only export metrics on /metrics. We can still allow to overwrite this for backwards compatibility.

What do you think?

@jack-berg
Copy link
Member

What about going in the prometheus client_java direction and only exporting on /metrics? What are the use cases for configuring a different path or multiple paths?

Some data points:

Note, to add a new environment variable like OTEL_EXPORTER_PROMETHEUS_PATH, the spec would have to be extended. Making that happen is a longshot because there's a moratorium on adding new environment variables, with the goal to develop / extend file configuration instead.

@breedx-splk
Copy link
Contributor

@fstab are you still looking to pick this up? It seems reasonable to me, but I have no idea if someone might be relying on the root path / or just "any" path. 🤷🏻

Having it be specific makes sense to me tho....and /metrics seems like a reasonable choice.

@fstab
Copy link
Member

fstab commented May 16, 2024

Thanks for the ping. Yes, I'll have a look, at the latest next week.

@fstab
Copy link
Member

fstab commented May 26, 2024

I created #6476.

@breedx-splk
Copy link
Contributor

This will be available in 1.39.0, approximately tomorrow. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants