Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modernise OpenTelemetry Collector example config #3518

Merged
merged 4 commits into from
Jan 9, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions docs/en/observability/apm/otel-direct.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -32,31 +32,31 @@ processors: <2>
batch:

exporters:
logging:
loglevel: warn <3>
debug:
verbosity: detailed <3>
otlp/elastic: <4>
# Elastic APM server https endpoint without the "https://" prefix
endpoint: "${ELASTIC_APM_SERVER_ENDPOINT}" <5> <7>
endpoint: "${env:ELASTIC_APM_SERVER_ENDPOINT}" <5> <7>
headers:
# Elastic APM Server secret token
Authorization: "Bearer ${ELASTIC_APM_SECRET_TOKEN}" <6> <7>
Authorization: "Bearer ${env:ELASTIC_APM_SECRET_TOKEN}" <6> <7>

service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging, otlp/elastic]
exporters: [debug, otlp/elastic]
metrics:
receivers: [otlp]
exporters: [logging, otlp/elastic]
exporters: [debug, otlp/elastic]
logs: <8>
receivers: [otlp]
exporters: [logging, otlp/elastic]
exporters: [debug, otlp/elastic]
----
<1> The receivers, like the
https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver[OTLP receiver], that forward data emitted by APM agents, or the https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver[host metrics receiver].
<2> We recommend using the https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md[Batch processor] and the https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiterprocessor/README.md[memory limiter processor]. For more information, see https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/README.md#recommended-processors[recommended processors].
<3> The https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/loggingexporter[logging exporter] is helpful for troubleshooting and supports various logging levels, like `debug`, `info`, `warn`, and `error`.
<3> The https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/debugexporter[debug exporter] is helpful for troubleshooting, and supports configurable verbosity levels: `basic` (default), `normal`, and `detailed`.
<4> Elastic {observability} endpoint configuration.
APM Server supports a ProtoBuf payload via both the OTLP protocol over gRPC transport {ot-grpc}[(OTLP/gRPC)]
and the OTLP protocol over HTTP transport {ot-http}[(OTLP/HTTP)].
Expand All @@ -65,13 +65,13 @@ https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otl
https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlpexporter[OTLP/gRPC exporter].
<5> Hostname and port of the APM Server endpoint. For example, `elastic-apm-server:8200`.
<6> Credential for Elastic APM <<secret-token,secret token authorization>> (`Authorization: "Bearer a_secret_token"`) or <<api-key,API key authorization>> (`Authorization: "ApiKey an_api_key"`).
<7> Environment-specific configuration parameters can be conveniently passed in as environment variables documented https://opentelemetry.io/docs/collector/configuration/#configuration-environment-variables[here] (e.g. `ELASTIC_APM_SERVER_ENDPOINT` and `ELASTIC_APM_SECRET_TOKEN`).
<7> Environment-specific configuration parameters can be conveniently passed in as environment variables documented https://opentelemetry.io/docs/collector/configuration/#environment-variables[here] (e.g. `ELASTIC_APM_SERVER_ENDPOINT` and `ELASTIC_APM_SECRET_TOKEN`).
<8> preview:[] To send OpenTelemetry logs to {stack} version 8.0+, declare a `logs` pipeline.

You're now ready to export traces and metrics from your services and applications.

TIP: When using the OpenTelemetry collector, you should always prefer sending data via the [`OTLP` exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter) to an Elastic APM Server.
Other methods, like using the [`elasticsearch` exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter) to send data directly to {es} will send data to the {stack},
TIP: When using the OpenTelemetry collector, you should always prefer sending data via the https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter[`OTLP` exporter] to an Elastic APM Server.
Other methods, like using the https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter[`elasticsearch` exporter] to send data directly to {es} will send data to the {stack},
but will bypass all of the validation and data processing that the APM Server performs.
In addition, your data will not be viewable in the {kib} {observability} apps if you use the `elasticsearch` exporter.

Expand Down Expand Up @@ -128,7 +128,7 @@ and <<open-telemetry-visualize,visualizing metrics>> in {kib}.
[[open-telemetry-proxy-apm]]
==== Proxy requests to APM Server

APM Server supports both the {ot-grpc}[(OTLP/gRPC)] and {ot-http}[(OTLP/HTTP)] protocol on the same port as Elastic APM agent requests. For ease of setup, we recommend using OTLP/HTTP when proxying or load balancing requests to the APM Server.
APM Server supports both the {ot-grpc}[OTLP/gRPC] and {ot-http}[OTLP/HTTP] protocol on the same port as Elastic APM agent requests. For ease of setup, we recommend using OTLP/HTTP when proxying or load balancing requests to the APM Server.

If you use the OTLP/gRPC protocol, requests to the APM Server must use either HTTP/2 over TLS or HTTP/2 Cleartext (H2C). No matter which protocol is used, OTLP/gRPC requests will have the header: `"Content-Type: application/grpc"`.

Expand Down