diff --git a/CHANGELOG.md b/CHANGELOG.md index 517e63ac1f1..daba7b5387e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,15 +11,106 @@ release. ### Traces +### Metrics + +### Logs + +### Resource + +### OpenTelemetry Protocol + +### Compatibility + +### SDK Configuration + +### Common + +### Supplementary Guidelines + +## v1.28.0 (2023-12-07) + +### Context + +- No changes. + +### Traces + +- Stabilize how exceptions are recorded using the Trace SDK. + ([#3769](https://github.com/open-telemetry/opentelemetry-specification/pull/3769)) +- Add definition for standard output span exporter. + ([#3740](https://github.com/open-telemetry/opentelemetry-specification/pull/3740)) + +### Metrics + +- Define experimental MetricFilter as a mechanism to filter collected metrics by the MetricReader + ([#3566](https://github.com/open-telemetry/opentelemetry-specification/pull/3566)) +- Add optional configuration for Prometheus exporters to promote resource attributes to metric attributes + ([#3761](https://github.com/open-telemetry/opentelemetry-specification/pull/3761)) +- Clarifications and flexibility in Exemplar speicification. + ([#3760](https://github.com/open-telemetry/opentelemetry-specification/pull/3760)) +- Add optional configuration for Prometheus exporters to optionally remove unit and type suffixes + ([#3777](https://github.com/open-telemetry/opentelemetry-specification/pull/3777)) + +### Logs + +- Add definition for standard output log record exporter. + ([#3741](https://github.com/open-telemetry/opentelemetry-specification/pull/3741)) +- BREAKING: Change `event.name` definition to include `namespace` and removed `event.domain` from log event attributes. + ([#3749](https://github.com/open-telemetry/opentelemetry-specification/pull/3749)) +- BREAKING: Refine the arguments of the emit Event API. Instead of accepting + a `LogRecord`, the individual arguments are enumerated along with the + implementation requirements on how those arguments map to `LogRecord`. + ([#3772](https://github.com/open-telemetry/opentelemetry-specification/pull/3772)) + +### Resource + +- No changes. + +### OpenTelemetry Protocol + +- Clarify HTTP endpoint configuration option handling. + ([#3739](https://github.com/open-telemetry/opentelemetry-specification/pull/3739)) + +### Compatibility + +- No changes. + +### SDK Configuration + +- Add `console` as an exporter type that is supported via environment variable configuration. + ([#3742](https://github.com/open-telemetry/opentelemetry-specification/pull/3742)) + +### Common + +- No changes. + +### Supplementary Guidelines + +- No changes. + +## v1.27.0 (2023-11-08) + +### Context + +- No changes. + +### Traces + - Add a new AddLink() operation to Span (experimental). ([#3678](https://github.com/open-telemetry/opentelemetry-specification/pull/3678)) ### Metrics +- No changes. + ### Logs +- No changes. + ### Resource +- No changes. + ### OpenTelemetry Protocol - New exporter implementations do not need to support @@ -28,10 +119,14 @@ release. ### Compatibility +- No changes. + ### SDK Configuration - Define file configuration parse and create operations. ([#3437](https://github.com/open-telemetry/opentelemetry-specification/pull/3437)) +- Add environment variable implementation guidelines. + ([#3738](https://github.com/open-telemetry/opentelemetry-specification/pull/3738)) ### Common @@ -40,6 +135,8 @@ release. ### Supplementary Guidelines +- No changes. + ## v1.26.0 (2023-10-10) ### Context diff --git a/experimental/trace/zpages.md b/experimental/trace/zpages.md index 14b29dec8c9..442c03babc3 100644 --- a/experimental/trace/zpages.md +++ b/experimental/trace/zpages.md @@ -24,7 +24,7 @@ zPages are an in-process alternative to external exporters. When included, they collect and aggregate tracing and metrics information in the background; this data is served on web pages when requested. -The idea of "zPages" originates from one of OpenTelemetry's predecessors, [OpenCensus](https://opencensus.io/). You can read more about zPages from the OpenCensus docs [here](https://opencensus.io/zpages) or the OTEP [here](https://github.com/open-telemetry/oteps/blob/main/text/0110-z-pages.md). OpenCensus has different zPage implementations in [Java](https://opencensus.io/zpages/java/), [Go](https://opencensus.io/zpages/go/), and [Node](https://opencensus.io/zpages/node/) and there has been similar internal solutions developed at companies like Uber. Within OpenTelemetry, zPages are being developed in C++. The OTel Collector also has [an implementation](https://github.com/open-telemetry/opentelemetry-collector/tree/master/extension/zpagesextension) of zPages. +The idea of "zPages" originates from one of OpenTelemetry's predecessors, [OpenCensus](https://opencensus.io/). You can read more about zPages from the OpenCensus docs [here](https://opencensus.io/zpages) or the OTEP [here](https://github.com/open-telemetry/oteps/blob/main/text/0110-z-pages.md). OpenCensus has different zPage implementations in [Java](https://opencensus.io/zpages/java/), [Go](https://opencensus.io/zpages/go/), and [Node](https://opencensus.io/zpages/node/) and there has been similar internal solutions developed at companies like Uber. Within OpenTelemetry, zPages are available in Go and Rust. The OTel Collector also has [an implementation](https://github.com/open-telemetry/opentelemetry-collector/tree/master/extension/zpagesextension) of zPages. zPages are uniquely useful in a couple of different ways. One is that they're more lightweight and quicker compared to installing external tracing systems like Jaeger and Zipkin, yet they still share useful ways to debug and gain insight into instrumented applications; these uses depend on the type of zPage, which is detailed below. For high throughput applications, zPages can also analyze more telemetry with the limited set of supported scenarios than external exporters; this is because zPages are in-memory while external exporters are typically configured to send a subset of telemetry for reach analysis to save costs. diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md index df304d1a316..abe1b80d33a 100644 --- a/spec-compliance-matrix.md +++ b/spec-compliance-matrix.md @@ -20,7 +20,7 @@ formats is required. Implementing more than one format is optional. | Create TracerProvider | | + | + | + | + | + | + | + | + | + | + | + | | Get a Tracer | | + | + | + | + | + | + | + | + | + | + | + | | Get a Tracer with schema_url | | + | + | | + | | | + | | + | | | -| Get a Tracer with scope attributes | | | | | | | | + | | | | | +| Get a Tracer with scope attributes | | | | | | | | + | | + | | | | Associate Tracer with InstrumentationScope | | | | | + | | | + | | | | | | Safe for concurrent calls | | + | + | + | + | + | + | + | + | + | + | + | | Shutdown (SDK only required) | | + | + | + | + | + | + | + | + | + | + | + | @@ -67,7 +67,7 @@ formats is required. Implementing more than one format is optional. | Unicode support for keys and string values | | + | + | + | + | + | + | + | + | + | + | + | | [Span linking](specification/trace/api.md#specifying-links) | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | | Links can be recorded on span creation | | + | + | | + | + | + | + | + | + | + | | -| Links can be recorded after span creation | | | | | | | | | | | | | +| Links can be recorded after span creation | | | | | | | | | | + | | | | Links order is preserved | | + | + | | + | + | + | + | + | + | + | | | [Span events](specification/trace/api.md#add-events) | | | | | | | | | | | | | | AddEvent | | + | + | + | + | + | + | + | + | + | + | + | @@ -102,7 +102,7 @@ formats is required. Implementing more than one format is optional. | It is possible to create any number of `MeterProvider`s. | X | + | + | + | + | | | + | + | + | + | | | `MeterProvider` provides a way to get a `Meter`. | | + | + | + | + | | | + | + | + | - | | | `get_meter` accepts name, `version` and `schema_url`. | | + | + | + | + | | | + | + | + | - | | -| `get_meter` accepts `attributes`. | | | | | | | | + | + | | | | +| `get_meter` accepts `attributes`. | | | | | | | | + | + | + | | | | When an invalid `name` is specified a working `Meter` implementation is returned as a fallback. | | + | + | + | + | | | | + | + | - | | | The fallback `Meter` `name` property keeps its original invalid value. | X | - | - | + | + | | | | + | - | - | | | Associate `Meter` with `InstrumentationScope`. | | | + | + | + | | | | + | + | | | @@ -212,6 +212,9 @@ formats is required. Implementing more than one format is optional. | The metrics SDK provides an `AlignedHistogramBucketExemplarReservoir` that is used by default for `ExplicitBucketHistogram` aggregation. | | | + | | - | | | | | | - | | | The metrics SDK provides an `ExemplarFilter` interface or extension point. | X | | - | | - | | | + | | | - | | | An `ExemplarFilter` has access to the measurement value, attributes, `Context` and timestamp. | X | | - | | - | | | + | | | - | | +| A metric Producer accepts an optional metric Filter | | | | | | | | | | | | | +| The metric Reader implementation supports registering metric Filter and passing them its registered metric Producers | | | | | | | | | | | | | +| The metric SDK's metric Producer implementations uses the metric Filter | | | | | | | | | | | | | ## Logs diff --git a/specification/baggage/api.md b/specification/baggage/api.md index ffccfe17fe8..fb6cb392ede 100644 --- a/specification/baggage/api.md +++ b/specification/baggage/api.md @@ -146,7 +146,7 @@ reasons. The API layer or an extension package MUST include the following `Propagator`s: -* A `TextMapPropagator` implementing the [W3C Baggage Specification](https://w3c.github.io/baggage). +* A `TextMapPropagator` implementing the [W3C Baggage Specification](https://www.w3.org/TR/baggage). See [Propagators Distribution](../context/api-propagators.md#propagators-distribution) for how propagators are to be distributed. diff --git a/specification/common/README.md b/specification/common/README.md index 1f17536bb64..1b46da3e628 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -53,9 +53,9 @@ indices that are kept in sync (e.g., two attributes `header_keys` and `header_va both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). -See [Attribute Naming](attribute-naming.md) for naming guidelines. +See [Attribute Naming](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/attribute-naming.md) for naming guidelines. -See [Requirement Level](attribute-requirement-level.md) for requirement levels guidelines. +See [Requirement Level](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/attribute-requirement-level.md) for requirement levels guidelines. See [this document](attribute-type-mapping.md) to find out how to map values obtained outside OpenTelemetry into OpenTelemetry attribute values. diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 548703db6f2..68f6ca6560f 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -1,156 +1,4 @@ # Attribute Naming -**Status**: [Stable](../document-status.md) - -
-Table of Contents - - - -- [Name Pluralization guidelines](#name-pluralization-guidelines) -- [Name Reuse Prohibition](#name-reuse-prohibition) -- [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) -- [Recommendations for Application Developers](#recommendations-for-application-developers) -- [otel.* Namespace](#otel-namespace) - - - -
- -_This section applies to Resource, Span, Log, and Metric attribute names (also -known as the "attribute keys"). For brevity within this section when we use the -term "name" without an adjective it is implied to mean "attribute name"._ - -Every name MUST be a valid Unicode sequence. - -_Note: we merely require that the names are represented as Unicode sequences. -This specification does not define how exactly the Unicode sequences are -encoded. The encoding can vary from one programming language to another and from -one wire format to another. Use the idiomatic way to represent Unicode in the -particular programming language or wire format._ - -Names SHOULD follow these rules: - -- Use namespacing to avoid name clashes. Delimit the namespaces using a dot - character. For example `service.version` denotes the service version where - `service` is the namespace and `version` is an attribute in that namespace. - -- Namespaces can be nested. For example `telemetry.sdk` is a namespace inside - top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute - inside `telemetry.sdk` namespace. - Note: the fact that an entity is located within another entity is typically - not a sufficient reason for forming nested namespaces. The purpose of a - namespace is to avoid name clashes, not to indicate entity hierarchies. This - purpose should primarily drive the decision about forming nested namespaces. - -- For each multi-word dot-delimited component of the attribute name separate the - words by underscores (i.e. use snake_case). For example `http.response.status_code` - denotes the status code in the http namespace. - -- Names SHOULD NOT coincide with namespaces. For example if - `service.instance.id` is an attribute name then it is no longer valid to have - an attribute named `service.instance` because `service.instance` is already a - namespace. Because of this rule be careful when choosing names: every existing - name prohibits existence of an equally named namespace in the future, and vice - versa: any existing namespace prohibits existence of an equally named - attribute key in the future. - -## Name Pluralization guidelines - -- When an attribute represents a single entity, the attribute name SHOULD be singular. - Examples: `host.name`, `db.user`, `container.id`. - -- When attribute can represent multiple entities, the attribute name SHOULD be pluralized - and the value type SHOULD be an array. E.g. `process.command_args` might include multiple - values: the executable name and command arguments. - -- When an attribute represents a measurement, - [Metric Name Pluralization Guidelines](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md#pluralization) - SHOULD be followed for the attribute name. - -## Name Reuse Prohibition - -A new attribute MUST NOT be added with the same name as an attribute that -existed in the past but was renamed (with a corresponding schema file). - -When introducing a new attribute name check all existing schema files to make -sure the name does not appear as a key of any "rename_attributes" section (keys -denote old attribute names in rename operations). - -## Recommendations for OpenTelemetry Authors - -- All names that are part of OpenTelemetry semantic conventions SHOULD be part - of a namespace. - -- When coming up with a new semantic convention make sure to check existing - namespaces ([Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md)) - to see if the new name fits. - -- When a new namespace is necessary consider whether it should be a top-level - namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). - -- Semantic conventions exist for four areas: for Resource, Span, Log, and Metric - attribute names. In addition, for spans we have two more areas: Event and Link - attribute names. Identical namespaces or names in all these areas MUST have - identical meanings. For example the `http.request.method` span attribute name denotes - exactly the same concept as the `http.request.method` metric attribute, has the same - data type and the same set of possible values (in both cases it records the - value of the HTTP protocol's request method as a string). - -- Semantic conventions MUST limit names to printable Basic Latin characters - (more precisely to - [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) - subset of Unicode code points). It is recommended to further limit names to - the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot - (as namespace delimiter). - -## Recommendations for Application Developers - -As an application developer when you need to record an attribute first consult -existing [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md). -If an appropriate name does not exists you will need to come up with a new name. -To do that consider a few options: - -- The name is specific to your company and may be possibly used outside the - company as well. To avoid clashes with names introduced by other companies (in - a distributed system that uses applications from multiple vendors) it is - recommended to prefix the new name by your company's reverse domain name, e.g. - `com.acme.shopname`. - -- The name is specific to your application that will be used internally only. If - you already have an internal company process that helps you to ensure no name - clashes happen then feel free to follow it. Otherwise it is recommended to - prefix the attribute name by your application name, provided that - the application name is reasonably unique within your organization (e.g. - `myuniquemapapp.longitude` is likely fine). Make sure the application name - does not clash with an existing semantic convention namespace. - -- It is not recommended to use existing OpenTelemetry semantic convention namespace - as a prefix for a new company- or application-specific attribute name. Doing so - may result in a name clash in the future, if OpenTelemetry decides to use that - same name for a different purpose or if some other third party instrumentation - decides to use that exact same attribute name and you combine that instrumentation - with your own. - -- The name may be generally applicable to applications in the industry. In that - case consider submitting a proposal to this specification to add a new name to - the semantic conventions, and if necessary also to add a new namespace. - -It is recommended to limit names to printable Basic Latin characters -(more precisely to -[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) -subset of Unicode code points). - -## otel.* Namespace - -Attribute names that start with `otel.` are reserved to be defined by -OpenTelemetry specification. These are typically used to express OpenTelemetry -concepts in formats that don't have a corresponding concept. - -For example, the `otel.scope.name` attribute is used to record the -instrumentation scope name, which is an OpenTelemetry concept that is natively -represented in OTLP, but does not have an equivalent in other telemetry formats -and protocols. - -Any additions to the `otel.*` namespace MUST be approved as part of -OpenTelemetry specification. +This page has moved to +[github.com/open-telemetry/semantic-conventions/docs/general/attribute-naming.md](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/attribute-naming.md). diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 7f510fe7313..bcbe62b0a84 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -1,76 +1,4 @@ # Attribute Requirement Levels for Semantic Conventions -**Status**: [Stable](../document-status.md) - -
-Table of Contents - - - -- [Required](#required) -- [Conditionally Required](#conditionally-required) -- [Recommended](#recommended) -- [Opt-In](#opt-in) -- [Performance suggestions](#performance-suggestions) - - - -
- -_This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ - -Attribute requirement levels apply to the [instrumentation library](../glossary.md#instrumentation-library). - -The following attribute requirement levels are specified: - -- [Required](#required) -- [Conditionally Required](#conditionally-required) -- [Recommended](#recommended) -- [Opt-In](#opt-in) - -The requirement level for an attribute is specified by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When specifying requirement levels, a semantic convention MUST take into account signal-specific requirements. - -For example, Metric attributes that may have high cardinality can only be defined with `Opt-In` level. - -A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. - - -For example, [Database semantic convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) references `network.transport` attribute defined in [General attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/README.md) with `Conditionally Required` level on it. - -## Required - -All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.request.method` is an example of a Required attribute. - -_Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ - -## Conditionally Required - -All instrumentations MUST populate the attribute when the given condition is satisfied. The semantic convention of a `Conditionally Required` attribute MUST clarify the condition under which the attribute is to be populated. - -`http.route` is an example of a conditionally required attribute that is populated when the instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. - -When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. - - -For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `network.peer.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. - -## Recommended - -Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. - -Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD allow for users to -opt-in to emit them as defined for the `Opt-In` requirement level (if the attributes are logically applicable). - -## Opt-In - -Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. - -This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled explicitly by a user making an informed decision. - -## Performance suggestions - -Here are several examples of expensive operations to be avoided by default: - -- DNS lookups to populate `server.address` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. -- forcing an `http.route` calculation before the HTTP framework calculates it -- reading response stream to find `http.response.body.size` when `Content-Length` header is not available +This page has moved to +[github.com/open-telemetry/semantic-conventions/docs/general/attribute-requirement-level.md](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/attribute-requirement-level.md). diff --git a/specification/common/url.md b/specification/common/url.md deleted file mode 100644 index bbe041ba67e..00000000000 --- a/specification/common/url.md +++ /dev/null @@ -1,45 +0,0 @@ -# Semantic conventions for URL - -**Status**: [Experimental](../document-status.md) - -This document defines semantic conventions that describe URL and its components. - -
-Table of Contents - - - -- [Attributes](#attributes) -- [Sensitive information](#sensitive-information) - - - -
- -## Attributes - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `https`; `ftp`; `telnet` | Recommended | -| `url.full` | string | Absolute URL describing a network resource according to [RFC3986](https://www.rfc-editor.org/rfc/rfc3986) [1] | `https://www.foo.bar/search?q=OpenTelemetry#SemConv`; `//localhost` | Recommended | -| `url.path` | string | The [URI path](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) component [2] | `/search` | Recommended | -| `url.query` | string | The [URI query](https://www.rfc-editor.org/rfc/rfc3986#section-3.4) component [3] | `q=OpenTelemetry` | Recommended | -| `url.fragment` | string | The [URI fragment](https://www.rfc-editor.org/rfc/rfc3986#section-3.5) component | `SemConv` | Recommended | - -**[1]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless. -`url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`. -`url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) and SHOULD NOT be validated or modified except for sanitizing purposes. - -**[2]:** When missing, the value is assumed to be `/` - -**[3]:** Sensitive content provided in query string SHOULD be scrubbed when instrumentations can identify it. - - -## Sensitive information - -Capturing URL and its components MAY impose security risk. User and password information, when they are provided in [User Information](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2.1) subcomponent, MUST NOT be recorded. - -Instrumentations that are aware of specific sensitive query string parameters MUST scrub their values before capturing `url.query` attribute. For example, native instrumentation of a client library that passes credentials or user location in URL, must scrub corresponding properties. - -_Note: Applications and telemetry consumers should scrub sensitive information from URL attributes on collected telemetry. In systems unable to identify sensitive information, certain attribute values may be redacted entirely._ diff --git a/specification/compatibility/prometheus_and_openmetrics.md b/specification/compatibility/prometheus_and_openmetrics.md index ecb74fc73dd..7f36beddf3c 100644 --- a/specification/compatibility/prometheus_and_openmetrics.md +++ b/specification/compatibility/prometheus_and_openmetrics.md @@ -385,11 +385,11 @@ OpenMetrics exemplar unless they would exceed the OpenMetrics In SDK Prometheus (pull) exporters, resource attributes SHOULD be converted to a single [`target_info` metric](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) -if the resource is not [empty](../resource/sdk.md#the-empty-resource); -otherwise, they MUST be dropped, and MUST NOT be attached as labels to other -metric families. The target_info metric MUST be an info-typed metric whose -labels MUST include the resource attributes, and MUST NOT include any other -labels. There MUST be at most one target_info metric exposed on an SDK +if the resource is not [empty](../resource/sdk.md#the-empty-resource). +The resource attributes MAY be copied to labels of exported metric families +if required by the exporter configuration, or MUST be dropped. The target_info metric MUST be an info-typed +metric whose labels MUST include the resource attributes, and MUST NOT include +any other labels. There MUST be at most one target_info metric exposed on an SDK Prometheus endpoint. In the Collector's Prometheus pull and push (remote-write) exporters, it is diff --git a/specification/configuration/sdk-environment-variables.md b/specification/configuration/sdk-environment-variables.md index 51803982d45..ef4794cb046 100644 --- a/specification/configuration/sdk-environment-variables.md +++ b/specification/configuration/sdk-environment-variables.md @@ -10,7 +10,18 @@ aliases: **Status**: [Mixed](../document-status.md) -The goal of this specification is to unify the environment variable names between different OpenTelemetry SDK implementations. SDKs MAY choose to allow configuration via the environment variables in this specification, but are not required to. If they do, they SHOULD use the names listed in this document. +The goal of this specification is to unify the environment variable names between different OpenTelemetry implementations. + +Implementations MAY choose to allow configuration via the environment variables in this specification, but are not required to. +If they do, they SHOULD use the names listed in this document. + +## Implementation guidelines + +**Status**: [Stable](../document-status.md) + +Environment variables MAY be handled (implemented) directly by a component, in the SDK, or in a separate component (e.g. environment-based autoconfiguration component). + +The environment-based configuration MUST have a direct code configuration equivalent. ## Parsing empty value @@ -25,7 +36,7 @@ The SDK MUST interpret an empty value of an environment variable the same way as ### Boolean value Any value that represents a Boolean MUST be set to true only by the case-insensitive string `"true"`, meaning `"True"` or `"TRUE"` are also accepted, as true. -An SDK MUST NOT extend this definition and define additional values that are interpreted as true. +An implementation MUST NOT extend this definition and define additional values that are interpreted as true. Any value not explicitly defined here as a true value, including unset and empty values, MUST be interpreted as false. If any value other than a true value, case-insensitive string `"false"`, empty, or unset is used, a warning SHOULD be logged to inform users about the fallback to false being applied. All Boolean environment variables SHOULD be named and defined such that false is the expected safe default behavior. @@ -33,17 +44,17 @@ Renaming or changing the default value MUST NOT happen without a major version u ### Numeric value -If an SDK chooses to support an integer-valued environment variable, it SHOULD support nonnegative values between 0 and 2³¹ − 1 (inclusive). Individual SDKs MAY choose to support a larger range of values. +If an implementation chooses to support an integer-valued environment variable, it SHOULD support nonnegative values between 0 and 2³¹ − 1 (inclusive). Individual SDKs MAY choose to support a larger range of values. > The following paragraph was added after stabilization and the requirements are -thus qualified as "SHOULD" to allow SDKs to avoid breaking changes. +thus qualified as "SHOULD" to allow implementations to avoid breaking changes. For new implementations, these should be treated as MUST requirements. -For variables accepting a numeric value, if the user provides a value the SDK cannot parse, -or which is outside the valid range for the configuration item, the SDK SHOULD +For variables accepting a numeric value, if the user provides a value the implementation cannot parse, +or which is outside the valid range for the configuration item, the implementation SHOULD generate a warning and gracefully ignore the setting, i.e., treat them as not set. -In particular, SDKs +In particular, implementations SHOULD NOT assign a custom interpretation e.g. to negative values if a negative value does not naturally apply to a configuration and does not have an explicitly specified meaning, but treat it like any other invalid value. @@ -57,15 +68,15 @@ because it might reset a value set from other configuration sources with the def ### Enum value -For variables which accept a known value out of a set, i.e., an enum value, SDK implementations MAY support additional values not listed here. -For variables accepting an enum value, if the user provides a value the SDK does not recognize, the SDK MUST generate a warning and gracefully ignore the setting. +For variables which accept a known value out of a set, i.e., an enum value, implementations MAY support additional values not listed here. +For variables accepting an enum value, if the user provides a value the implementation does not recognize, the implementation MUST generate a warning and gracefully ignore the setting. If a null object (empty, no-op) value is acceptable, then the enum value representing it MUST be `"none"`. ### Duration Any value that represents a duration, for example a timeout, MUST be an integer representing a number of -milliseconds. The value is non-negative - if a negative value is provided, the SDK MUST generate a warning, +milliseconds. The value is non-negative - if a negative value is provided, the implementation MUST generate a warning, gracefully ignore the setting and use the default value if it is defined. For example, the value `12000` indicates 12000 milliseconds, i.e., 12 seconds. @@ -82,7 +93,7 @@ For example, the value `12000` indicates 12000 milliseconds, i.e., 12 seconds. | OTEL_LOG_LEVEL | Log level used by the SDK logger | "info" | | | OTEL_PROPAGATORS | Propagators to be used as a comma-separated list | "tracecontext,baggage" | Values MUST be deduplicated in order to register a `Propagator` only once. | | OTEL_TRACES_SAMPLER | Sampler to be used for traces | "parentbased_always_on" | See [Sampling](../trace/sdk.md#sampling) | -| OTEL_TRACES_SAMPLER_ARG | String value to be used as the sampler argument | | The specified value will only be used if OTEL_TRACES_SAMPLER is set. Each Sampler type defines its own expected input, if any. Invalid or unrecognized input MUST be logged and MUST be otherwise ignored, i.e. the SDK MUST behave as if OTEL_TRACES_SAMPLER_ARG is not set. | +| OTEL_TRACES_SAMPLER_ARG | String value to be used as the sampler argument | | The specified value will only be used if OTEL_TRACES_SAMPLER is set. Each Sampler type defines its own expected input, if any. Invalid or unrecognized input MUST be logged and MUST be otherwise ignored, i.e. the implementation MUST behave as if OTEL_TRACES_SAMPLER_ARG is not set. | Known values for `OTEL_PROPAGATORS` are: @@ -140,7 +151,7 @@ Depending on the value of `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_SAMPLER_ARG` may b ## Attribute Limits -SDKs SHOULD only offer environment variables for the types of attributes, for +Implementations SHOULD only offer environment variables for the types of attributes, for which that SDK implements truncation mechanism. See the SDK [Attribute Limits](../common/README.md#attribute-limits) section for the definition of the limits. @@ -219,23 +230,32 @@ We define environment variables for setting one or more exporters per signal. | OTEL_METRICS_EXPORTER | Metrics exporter to be used | "otlp" | | OTEL_LOGS_EXPORTER | Logs exporter to be used | "otlp" | -The SDK MAY accept a comma-separated list to enable setting multiple exporters. +The implementation MAY accept a comma-separated list to enable setting multiple exporters. Known values for `OTEL_TRACES_EXPORTER` are: - `"otlp"`: [OTLP](../protocol/otlp.md) - `"zipkin"`: [Zipkin](https://zipkin.io/zipkin-api/) (Defaults to [protobuf](https://github.com/openzipkin/zipkin-api/blob/master/zipkin.proto) format) +- `"console"`: [Standard Output](../trace/sdk_exporters/stdout.md) +- `"logging"`: [Standard Output](../trace/sdk_exporters/stdout.md). It is a deprecated value left for backwards compatibility. It SHOULD +NOT be supported by new implementations. - `"none"`: No automatically configured exporter for traces. Known values for `OTEL_METRICS_EXPORTER` are: - `"otlp"`: [OTLP](../protocol/otlp.md) - `"prometheus"`: [Prometheus](https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md) +- `"console"`: [Standard Output](../metrics/sdk_exporters/stdout.md) +- `"logging"`: [Standard Output](../metrics/sdk_exporters/stdout.md). It is a deprecated value left for backwards compatibility. It SHOULD +NOT be supported by new implementations. - `"none"`: No automatically configured exporter for metrics. Known values for `OTEL_LOGS_EXPORTER` are: - `"otlp"`: [OTLP](../protocol/otlp.md) +- `"console"`: [Standard Output](../logs/sdk_exporters/stdout.md) +- `"logging"`: [Standard Output](../logs/sdk_exporters/stdout.md). It is a deprecated value left for backwards compatibility. It SHOULD +NOT be supported by new implementations. - `"none"`: No automatically configured exporter for logs. ## Metrics SDK Configuration diff --git a/specification/context/api-propagators.md b/specification/context/api-propagators.md index a1c85ee0f6c..762b309ef01 100644 --- a/specification/context/api-propagators.md +++ b/specification/context/api-propagators.md @@ -333,9 +333,9 @@ Required parameters: The official list of propagators that MUST be maintained by the OpenTelemetry organization and MUST be distributed as OpenTelemetry extension packages: -* [W3C TraceContext](https://www.w3.org/TR/trace-context/). MAY alternatively +* [W3C TraceContext](https://www.w3.org/TR/trace-context). MAY alternatively be distributed as part of the OpenTelemetry API. -* [W3C Baggage](https://w3c.github.io/baggage). MAY alternatively +* [W3C Baggage](https://www.w3.org/TR/baggage). MAY alternatively be distributed as part of the OpenTelemetry API. * [B3](https://github.com/openzipkin/b3-propagation). * [Jaeger](https://www.jaegertracing.io/docs/latest/client-libraries/#propagation-format). diff --git a/specification/logs/README.md b/specification/logs/README.md index b0c3fb5acbe..eaf1ab28212 100644 --- a/specification/logs/README.md +++ b/specification/logs/README.md @@ -284,7 +284,7 @@ auto-instrumenting solutions that modify trace logging libraries used by the application to automatically output the trace context such as the trace id or span id with every log statement. The trace context can be automatically extracted from incoming requests if standard compliant request propagation is -used, e.g. via [W3C TraceContext](https://w3c.github.io/trace-context/). In +used, e.g. via [W3C TraceContext](https://www.w3.org/TR/trace-context). In addition, the requests outgoing from the application may be injected with the same trace context data, thus resulting in context propagation through the application and creating an opportunity to have full trace context in logs diff --git a/specification/logs/event-api.md b/specification/logs/event-api.md index 6c59ac226b3..b377c704e20 100644 --- a/specification/logs/event-api.md +++ b/specification/logs/event-api.md @@ -14,6 +14,7 @@ * [EventLogger Operations](#eventlogger-operations) + [Create EventLogger](#create-eventlogger) + [Emit Event](#emit-event) +- [Optional and required parameters](#optional-and-required-parameters) @@ -30,11 +31,11 @@ From OpenTelemetry's perspective LogRecords and Events are both represented using the same [data model](./data-model.md). However, OpenTelemetry does recognize a subtle semantic difference between -LogRecords and Events: Events are LogRecords which have a `name` and `domain`. -Within a particular `domain`, the `name` uniquely defines a particular class or -type of event. Events with the same `domain` / `name` follow the same schema -which assists in analysis in observability platforms. Events are described in -more detail in the [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/events.md). +LogRecords and Events: Events are LogRecords which have a `name` which uniquely +defines a particular class or type of event. All events with the same `name` +have `Payloads` that conform to the same schema, which assists in analysis in +observability platforms. Events are described in more detail in +the [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/events.md). While the logging space has a diverse legacy with many existing logging libraries in different languages, there is not ubiquitous alignment with @@ -65,8 +66,6 @@ on `EventLogger`. * `logger` - the delegate [Logger](./bridge-api.md#logger) used to emit `Events` as `LogRecord`s. -* `event_domain` - the domain of emitted events, used to set the `event.domain` - attribute. #### Emit Event @@ -76,19 +75,55 @@ This function MAY be named `logEvent`. **Parameters:** -* `event_name` - the Event name. This argument MUST be recorded as a `LogRecord` - attribute with the key `event.name`. Care MUST be taken by the implementation - to not override or delete this attribute while the Event is emitted to - preserve its identity. -* `logRecord` - the [LogRecord](./data-model.md#log-and-event-record-definition) representing the Event. +* The `Name` of the Event, as described + in [event.name semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/events.md). +* The (`AnyValue`) (optional) `Payload` of the Event. +* The `Timestamp` (optional) of the Event. +* The [Context](../context/README.md) (optional) associated with the Event. +* The `SeverityNumber` (optional) of the Event. +* The `Attributes` (optional) of the Event. Event `Attributes` provide + additional details about the Event which are not part of the + well-defined `Payload`. **Implementation Requirements:** -The implementation MUST [emit](./bridge-api.md#emit-a-logrecord) the `logRecord` to -the `logger` specified when [creating the EventLogger](#create-eventlogger) -after making the following changes: - -* The `event_domain` specified - when [creating the EventLogger](#create-eventlogger) MUST be set as - the `event.domain` attribute on the `logRecord`. -* The `event_name` MUST be set as the `event.name` attribute on the `logRecord`. +The implementation MUST use the parameters +to [emit a logRecord](./bridge-api.md#emit-a-logrecord) using the `logger` +specified when [creating the EventLogger](#create-eventlogger) as follows: + +* The `Name` MUST be used to set + the `event.name` [Attribute](./data-model.md#field-attributes). If + the `Attributes` provided by the user contain an `event.name` attribute the + value provided in the `Name` takes precedence. +* If provided by the user, the `Payload` MUST be used to set + the [Body](./data-model.md#field-body). If not provided, `Body` MUST not be + set. +* If provided by the user, the `Timestamp` MUST be used to set + the [Timestamp](./data-model.md#field-timestamp). If not provided, `Timestamp` + MUST be set to the current time when [emit](#emit-event) was called. +* The [Observed Timestamp](./data-model.md#field-observedtimestamp) MUST not be + set. (NOTE: [emit a logRecord](./bridge-api.md#emit-a-logrecord) will + set `ObservedTimestamp` to the current time when unset.) +* If provided by the user, the `Context` MUST be used to set + the [Context](./bridge-api.md#emit-a-logrecord). If not provided, `Context` + MUST be set to the current Context. +* If provided by the user, the `SeverityNumber` MUST be used to set + the [Severity Number](./data-model.md#field-severitynumber) when emitting the + logRecord. If not provided, `SeverityNumber` MUST be set + to `SEVERITY_NUMBER_INFO=9`. +* The [Severity Text](./data-model.md#field-severitytext) MUST not be set. +* If provided by the user, the `Attributes` MUST be used to set + the [Attributes](./data-model.md#field-attributes). The user + provided `Attributes` MUST not take over the `event.name` + attribute previously discussed. + +## Optional and required parameters + +The operations defined include various parameters, some of which are marked +optional. Parameters not marked optional are required. + +For each optional parameter, the API MUST be structured to accept it, but MUST +NOT obligate a user to provide it. + +For each required parameter, the API MUST be structured to obligate a user to +provide it. diff --git a/specification/logs/sdk_exporters/stdout.md b/specification/logs/sdk_exporters/stdout.md new file mode 100644 index 00000000000..63efd060f44 --- /dev/null +++ b/specification/logs/sdk_exporters/stdout.md @@ -0,0 +1,22 @@ + + +# OpenTelemetry LogRecord Exporter - Standard output + +**Status**: [Experimental](../../document-status.md) + +"Standard output" LogRecord Exporter is a [LogRecord +Exporter](../sdk.md#logrecordexporter) which outputs the logs to +stdout/console. + +[OpenTelemetry SDK](../../overview.md#sdk) authors MAY choose the best idiomatic +name for their language. For example, ConsoleExporter, StdoutExporter, +StreamExporter, etc. + +If a language provides a mechanism to automatically configure a +[LogRecordProcessor](../sdk.md#logrecordprocessor) to pair with the associated +exporter (e.g., using the [`OTEL_LOGS_EXPORTER` environment +variable](../../configuration/sdk-environment-variables.md#exporter-selection)), by +default the standard output exporter SHOULD be paired with a [simple +processor](../sdk.md#simple-processor). diff --git a/specification/metrics/README.md b/specification/metrics/README.md index 82b94af96d8..8fad95d83a7 100644 --- a/specification/metrics/README.md +++ b/specification/metrics/README.md @@ -30,7 +30,7 @@ Given there are many well-established metrics solutions that exist today, it is important to understand the goals of OpenTelemetry’s metrics effort: * **Being able to connect metrics to other signals**. For example, metrics and - traces can be correlated via exemplars, and metrics attributes can be enriched + traces can be correlated via [exemplars](data-model.md#exemplars), and metrics attributes can be enriched via [Baggage](../baggage/api.md) and [Context](../context/README.md). Additionally, [Resource](../resource/sdk.md) can be applied to [logs](../overview.md#log-signal)/[metrics](../overview.md#metric-signal)/[traces](../overview.md#tracing-signal) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 0ac9ad9f29e..47fc44fda3b 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -54,6 +54,7 @@ linkTitle: SDK + [AlwaysOn](#alwayson) + [AlwaysOff](#alwaysoff) + [TraceBased](#tracebased) + + [Configuration](#configuration-2) * [ExemplarReservoir](#exemplarreservoir) * [Exemplar defaults](#exemplar-defaults) + [SimpleFixedSizeExemplarReservoir](#simplefixedsizeexemplarreservoir) @@ -73,7 +74,11 @@ linkTitle: SDK * [Pull Metric Exporter](#pull-metric-exporter) - [MetricProducer](#metricproducer) * [Interface Definition](#interface-definition-1) - + [Produce() batch](#produce-batch) + + [Produce batch](#produce-batch) +- [MetricFilter](#metricfilter) + * [Interface Definition](#interface-definition-2) + + [TestMetric](#testmetric) + + [TestAttributes](#testattributes) - [Defaults and configuration](#defaults-and-configuration) - [Numerical limits handling](#numerical-limits-handling) - [Compatibility requirements](#compatibility-requirements) @@ -972,11 +977,21 @@ Using this ExemplarFilter is as good as disabling Exemplar feature. An ExemplarFilter which makes those measurements eligible for being an Exemplar, which are recorded in the context of a sampled parent span. +#### Configuration + +The ExemplarFilter SHOULD be a configuration parameter of a `MeterProvider` for +an SDK. The default value SHOULD be `TraceBased`. The filter configuration +SHOULD follow the [environment variable specification](../configuration/sdk-environment-variables.md#exemplar). + ### ExemplarReservoir The `ExemplarReservoir` interface MUST provide a method to offer measurements to the reservoir and another to collect accumulated Exemplars. +A new `ExemplarReservoir` MUST be created for every known timeseries data point, +as determined by aggregation and view configuration. This data point, and its +set of defining attributes, are referred to as the associated timeseries point. + The "offer" method SHOULD accept measurements, including: - The `value` of the measurement. @@ -993,18 +1008,26 @@ span context and baggage can be inspected at this point. The "offer" method does not need to store all measurements it is given and MAY further sample beyond the `ExemplarFilter`. +The "offer" method MAY accept a filtered subset of `Attributes` which diverge +from the timeseries the reservoir is associated with. This MUST be clearly +documented in the API interface and the reservoir MUST be given the `Attributes` +associated with its timeseries point either at construction so that additional +sampling performed by the reservoir has access to all attributes from a +measurement in the "offer" method. SDK authors are encouraged to benchmark +whether this option works best for their implementation. + The "collect" method MUST return accumulated `Exemplar`s. Exemplars are expected to abide by the `AggregationTemporality` of any metric point they are recorded with. In other words, Exemplars reported against a metric data point SHOULD have -occurred within the start/stop timestamps of that point. SDKs are free to +occurred within the start/stop timestamps of that point. SDKs are free to decide whether "collect" should also reset internal storage for delta temporal aggregation collection, or use a more optimal implementation. `Exemplar`s MUST retain any attributes available in the measurement that -are not preserved by aggregation or view configuration. Specifically, at a -minimum, joining together attributes on an `Exemplar` with those available -on its associated metric data point should result in the full set of attributes -from the original sample measurement. +are not preserved by aggregation or view configuration for the associated +timeseries. Joining together attributes on an `Exemplar` with +those available on its associated metric data point should result in the +full set of attributes from the original sample measurement. The `ExemplarReservoir` SHOULD avoid allocations when sampling exemplars. @@ -1015,9 +1038,15 @@ The SDK SHOULD include two types of built-in exemplar reservoirs: 1. `SimpleFixedSizeExemplarReservoir` 2. `AlignedHistogramBucketExemplarReservoir` -By default, explicit bucket histogram aggregation with more than 1 bucket will -use `AlignedHistogramBucketExemplarReservoir`. All other aggregations will use -`SimpleFixedSizeExemplarReservoir`. +By default: + +- Explicit bucket histogram aggregation with more than 1 bucket will +use `AlignedHistogramBucketExemplarReservoir`. +- Base2 Exponential Histogram Aggregation SHOULD use a + `SimpleFixedSizeExemplarReservoir` with a reservoir equal to the + smaller of the maximum number of buckets configured on the aggregation or + twenty (e.g. `min(20, max_buckets)`). +- All other aggregations will use `SimpleFixedSizeExemplarReservoir`. #### SimpleFixedSizeExemplarReservoir @@ -1027,7 +1056,11 @@ measurements should be sampled. For example, the [simple reservoir sampling algorithm](https://en.wikipedia.org/wiki/Reservoir_sampling) can be used: ``` - bucket = random_integer(0, num_measurements_seen) + if num_measurements_seen < num_buckets then + bucket = num_measurements_seen + else + bucket = random_integer(0, num_measurements_seen) + end if bucket < num_buckets then reservoir[bucket] = measurement end @@ -1038,8 +1071,9 @@ cycle. For the above example, that would mean that the `num_measurements_seen` count is reset every time the reservoir is collected. This Exemplar reservoir MAY take a configuration parameter for the size of the -reservoir pool. If no size configuration is provided, the default size of `1` -SHOULD be used. +reservoir. If no size configuration is provided, the default size MAY be +the number of possible concurrent threads (e.g. numer of CPUs) to help reduce +contention. Otherwise, a default size of `1` SHOULD be used. #### AlignedHistogramBucketExemplarReservoir @@ -1083,8 +1117,12 @@ SHOULD provide at least the following: * The default output `aggregation` (optional), a function of instrument kind. If not configured, the [default aggregation](#default-aggregation) SHOULD be used. * The default output `temporality` (optional), a function of instrument kind. If not configured, the Cumulative temporality SHOULD be used. * **Status**: [Experimental](../document-status.md) - The default aggregation cardinality limit to use, a function of instrument kind. If not configured, a default value of 2000 SHOULD be used. +* **Status**: [Experimental](../document-status.md) - The [MetricFilter](#metricfilter) to apply to metrics and attributes during `MetricReader#Collect`. * Zero of more [MetricProducer](#metricproducer)s (optional) to collect metrics from in addition to metrics from the SDK. +**Status**: [Experimental](../document-status.md) - A `MetricReader` SHOULD provide the [MetricFilter](#metricfilter) to the SDK or registered [MetricProducer](#metricproducer)(s) +when calling the `Produce` operation. + The [MetricReader.Collect](#collect) method allows general-purpose `MetricExporter` instances to explicitly initiate collection, commonly used with pull-based metrics collection. A common implementation of @@ -1472,10 +1510,10 @@ modeled to interact with other components in the SDK: ## MetricProducer -**Status**: [Stable](../document-status.md) +**Status**: [Stable](../document-status.md) except where otherwise specified `MetricProducer` defines the interface which bridges to third-party metric -sources MUST implement so they can be plugged into an OpenTelemetry +sources MUST implement, so they can be plugged into an OpenTelemetry [MetricReader](#metricreader) as a source of aggregated metric data. The SDK's in-memory state MAY implement the `MetricProducer` interface for convenience. @@ -1505,10 +1543,14 @@ bridge pre-processed data. A `MetricProducer` MUST support the following functions: -#### Produce() batch +#### Produce batch `Produce` provides metrics from the MetricProducer to the caller. `Produce` -MUST return a batch of [Metric points](./data-model.md#metric-points). +MUST return a batch of [Metric points](./data-model.md#metric-points), filtered by the optional +`metricFilter` parameter. Implementation SHOULD use the filter as early as +possible to gain as much performance gain possible (memory allocation, +internal metric fetching, etc). + If the batch of [Metric points](./data-model.md#metric-points) includes resource information, `Produce` SHOULD require a resource as a parameter. `Produce` does not have any other required parameters, however, [OpenTelemetry @@ -1525,6 +1567,80 @@ If a batch of [Metric points](./data-model.md#metric-points) can include `Produce` SHOULD include a single InstrumentationScope which identifies the `MetricProducer`. +**Parameters:** + +**Status**: [Experimental](../document-status.md) `metricFilter`: An optional [MetricFilter](#metricfilter). + +## MetricFilter + +**Status**: [Experimental](../document-status.md) + +`MetricFilter` defines the interface which enables the [MetricReader](#metricreader)'s +registered [MetricProducers](#metricproducer) or the SDK's [MetricProducer](#metricproducer) to filter aggregated data points +([Metric points](./data-model.md#metric-points)) inside its `Produce` operation. +The filtering is done at the [MetricProducer](#metricproducer) for performance reasons. + +The `MetricFilter` allows filtering an entire metric stream - dropping or allowing all its attribute sets - +by its `TestMetric` operation, which accepts the metric stream information +(scope, name, kind and unit) and returns an enumeration: `Accept`, `Drop` +or `Allow_Partial`. If the latter returned, the `TestAttributes` operation +is to be called per attribute set of that metric stream, returning an enumeration +determining if the data point for that (metric stream, attributes) pair is to be +allowed in the result of the [MetricProducer](#metricproducer) `Produce` operation. + +### Interface Definition + +A `MetricFilter` MUST support the following functions: + +#### TestMetric + +This operation is called once for every metric stream, in each [MetricProducer](#metricproducer) `Produce` +operation. + +**Parameters:** + +- `instrumentationScope`: the metric stream instrumentation scope +- `name`: the name of the metric stream +- `kind`: the metric stream [kind](./data-model.md#point-kinds) +- `unit`: the metric stream unit + +Returns: `MetricFilterResult` + +`MetricFilterResult` is one of: + +* `Accept` - All attributes of the given metric stream are allowed (not to be filtered). + This provides a "short-circuit" as there is no need to call `TestAttributes` operation + for each attribute set. +* `Drop` - All attributes of the given metric stream are NOT allowed (filtered out - dropped). + This provides a "short-circuit" as there is no need to call `TestAttributes` operation + for each attribute set, and no need to collect those data points be it synchronous or asynchronous: + e.g. the callback for this given instrument does not need to be invoked. +* `Accept_Partial` - Some attributes are allowed and some aren't, hence `TestAttributes` + operation must be called for each attribute set of that instrument. + +#### TestAttributes + +An operation which determines for a given metric stream and attribute set if it should be allowed +or filtered out. + +This operation should only be called if `TestMetric` operation returned `Accept_Partial` for +the given metric stream arguments (`instrumentationScope`, `name`, `kind`, `unit`). + +**Parameters:** + +- `instrumentationScope`: the metric stream instrumentation scope +- `name`: the name of the metric stream +- `kind`: the metric stream kind +- `unit`: the metric stream unit +- `attributes`: the attributes + +Returns: `AttributesFilterResult` + +`AttributesFilterResult` is one of: + +* `Accept` - This given `attributes` are allowed (not to be filtered). +* `Drop` - This given `attributes` are NOT allowed (filtered out - dropped). + ## Defaults and configuration The SDK MUST provide configuration according to the [SDK environment diff --git a/specification/metrics/sdk_exporters/prometheus.md b/specification/metrics/sdk_exporters/prometheus.md index 8470a55694d..e67df6f4d31 100644 --- a/specification/metrics/sdk_exporters/prometheus.md +++ b/specification/metrics/sdk_exporters/prometheus.md @@ -25,3 +25,15 @@ A Prometheus Exporter MAY support [OpenMetrics Text Format](https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#openmetrics-text-format), including the [Exemplars](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#exemplars). + +A Prometheus Exporter MAY offer configuration to add resource attributes as metric attributes. +By default, it MUST NOT add any resource attributes as metric attributes. +The configuration SHOULD allow the user to select which resource attributes to copy (e.g. +include / exclude or regular expression based). Copied Resource attributes MUST NOT be +excluded from target_info. + +A Prometheus Exporter MAY support a configuration option to produce metrics without a [unit suffix](../../compatibility/prometheus_and_openmetrics.md#metric-metadata) +or UNIT metadata. The option MAY be named `without_units`, and MUST be `false` by default. + +A Prometheus Exporter MAY support a configuration option to produce metrics without a [type suffix](../../compatibility/prometheus_and_openmetrics.md#metric-metadata). +The option MAY be named `without_type_suffix`, and MUST be `false` by default. diff --git a/specification/protocol/exporter.md b/specification/protocol/exporter.md index a7d5a02fa21..22cf934c613 100644 --- a/specification/protocol/exporter.md +++ b/specification/protocol/exporter.md @@ -10,13 +10,24 @@ This document specifies the configuration options available to the OpenTelemetry ## Configuration Options -The following configuration options MUST be available to configure the OTLP exporter. Each configuration option MUST be overridable by a signal specific option. +The following configuration options MUST be available to configure the OTLP exporter. +Each configuration option MUST be overridable by a signal specific option. -- **Endpoint (OTLP/HTTP)**: Target URL to which the exporter is going to send spans or metrics. The endpoint MUST be a valid URL with scheme (http or https) and host, MAY contain a port, SHOULD contain a path and MUST NOT contain other parts (such as query string or fragment). A scheme of https indicates a secure connection. When using `OTEL_EXPORTER_OTLP_ENDPOINT`, exporters MUST construct per-signal URLs as [described below](#endpoint-urls-for-otlphttp). The per-signal endpoint configuration options take precedence and can be used to override this behavior (the URL is used as-is for them, without any modifications). See the [OTLP Specification][otlphttp-req] for more details. +- **Endpoint (OTLP/HTTP)**: Target URL to which the exporter is going to send spans, metrics, or logs. + The implementation MUST honor the following [URL components](https://datatracker.ietf.org/doc/html/rfc3986#section-3): + - scheme (`http` or `https`) + - host + - port + - path + + The implementation MAY ignore all other URL components. + + A scheme of `https` indicates a secure connection. + When using `OTEL_EXPORTER_OTLP_ENDPOINT`, exporters MUST construct per-signal URLs as [described below](#endpoint-urls-for-otlphttp). The per-signal endpoint configuration options take precedence and can be used to override this behavior (the URL is used as-is for them, without any modifications). See the [OTLP Specification][otlphttp-req] for more details. - Default: `http://localhost:4318` [1] - Env vars: `OTEL_EXPORTER_OTLP_ENDPOINT` `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` -- **Endpoint (OTLP/gRPC)**: Target to which the exporter is going to send spans or metrics. The endpoint SHOULD accept any form allowed by the underlying gRPC client implementation. Additionally, the endpoint MUST accept a URL with a scheme of either `http` or `https`. A scheme of `https` indicates a secure connection and takes precedence over the `insecure` configuration setting. A scheme of `http` indicates an insecure connection and takes precedence over the `insecure` configuration setting. If the gRPC client implementation does not support an endpoint with a scheme of `http` or `https` then the endpoint SHOULD be transformed to the most sensible format for that implementation. +- **Endpoint (OTLP/gRPC)**: Target to which the exporter is going to send spans, metrics, or logs. The option SHOULD accept any form allowed by the underlying gRPC client implementation. Additionally, the option MUST accept a URL with a scheme of either `http` or `https`. A scheme of `https` indicates a secure connection and takes precedence over the `insecure` configuration setting. A scheme of `http` indicates an insecure connection and takes precedence over the `insecure` configuration setting. If the gRPC client implementation does not support an endpoint with a scheme of `http` or `https` then the endpoint SHOULD be transformed to the most sensible format for that implementation. - Default: `http://localhost:4317` [1] - Env vars: `OTEL_EXPORTER_OTLP_ENDPOINT` `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` @@ -137,7 +148,7 @@ relative to `http://collector:4318/mycollector/`. ### Specify Protocol -The `OTEL_EXPORTER_OTLP_PROTOCOL`, `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`, and `OTEL_EXPORTER_OTLP_METRICS_PROTOCOL` environment variables specify the OTLP transport protocol. Supported values: +The `OTEL_EXPORTER_OTLP_PROTOCOL`, `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`, `OTEL_EXPORTER_OTLP_METRICS_PROTOCOL` and `OTEL_EXPORTER_OTLP_LOGS_PROTOCOL` environment variables specify the OTLP transport protocol. Supported values: - `grpc` for protobuf-encoded data using gRPC wire format over HTTP/2 connection - `http/protobuf` for protobuf-encoded data over HTTP connection @@ -154,7 +165,7 @@ release). ### Specifying headers via environment variables -The `OTEL_EXPORTER_OTLP_HEADERS`, `OTEL_EXPORTER_OTLP_TRACES_HEADERS`, `OTEL_EXPORTER_OTLP_METRICS_HEADERS` environment variables will contain a list of key value pairs, and these are expected to be represented in a format matching to the [W3C Correlation-Context](https://github.com/w3c/baggage/blob/master/baggage/HTTP_HEADER_FORMAT.md), except that additional semi-colon delimited metadata is not supported, i.e.: key1=value1,key2=value2. All attribute values MUST be considered strings. +The `OTEL_EXPORTER_OTLP_HEADERS`, `OTEL_EXPORTER_OTLP_TRACES_HEADERS`, `OTEL_EXPORTER_OTLP_METRICS_HEADERS`, `OTEL_EXPORTER_OTLP_LOGS_HEADERS` environment variables will contain a list of key value pairs, and these are expected to be represented in a format matching to the [W3C Baggage](https://www.w3.org/TR/baggage/#header-content), except that additional semi-colon delimited metadata is not supported, i.e.: key1=value1,key2=value2. All attribute values MUST be considered strings. ## Retry diff --git a/specification/resource/sdk.md b/specification/resource/sdk.md index 86e83ee215f..2c3df621323 100644 --- a/specification/resource/sdk.md +++ b/specification/resource/sdk.md @@ -137,7 +137,7 @@ has higher priority. The `OTEL_RESOURCE_ATTRIBUTES` environment variable will contain of a list of key value pairs, and these are expected to be represented in a format matching -to the [W3C Baggage](https://w3c.github.io/baggage), except that additional +to the [W3C Baggage](https://www.w3.org/TR/baggage/#header-content), except that additional semi-colon delimited metadata is not supported, i.e.: `key1=value1,key2=value2`. All attribute values MUST be considered strings and characters outside the `baggage-octet` range MUST be percent-encoded. diff --git a/specification/trace/exceptions.md b/specification/trace/exceptions.md index 303f3b04234..8ede02b5886 100644 --- a/specification/trace/exceptions.md +++ b/specification/trace/exceptions.md @@ -1,6 +1,6 @@ # Exceptions -**Status**: [Experimental](../document-status.md) +**Status**: [Stable](../document-status.md) This document defines how to record exceptions and their required attributes. diff --git a/specification/trace/sdk_exporters/stdout.md b/specification/trace/sdk_exporters/stdout.md new file mode 100644 index 00000000000..0ec93f343b5 --- /dev/null +++ b/specification/trace/sdk_exporters/stdout.md @@ -0,0 +1,22 @@ + + +# OpenTelemetry Span Exporter - Standard output + +**Status**: [Stable](../../document-status.md) + +"Standard output" Span Exporter is a [Span +Exporter](../sdk.md#span-exporter) which outputs the spans to +stdout/console. + +[OpenTelemetry SDK](../../overview.md#sdk) authors MAY choose the best idiomatic +name for their language. For example, ConsoleExporter, StdoutExporter, +StreamExporter, LoggingExporter etc. + +If a language provides a mechanism to automatically configure a +[Span processor](../sdk.md#span-processor) to pair with the associated +exporter (e.g., using the [`OTEL_TRACES_EXPORTER` environment +variable](../../configuration/sdk-environment-variables.md#exporter-selection)), by +default the standard output exporter SHOULD be paired with a [simple +processor](../sdk.md#simple-processor). diff --git a/specification/versioning-and-stability.md b/specification/versioning-and-stability.md index 33d7a2aafa9..b2632067aad 100644 --- a/specification/versioning-and-stability.md +++ b/specification/versioning-and-stability.md @@ -212,7 +212,6 @@ Semantic Conventions defines the set of fields in the OTLP data model: - For log records that are [Log Events](logs/event-api.md) - The following data provided to [emit event](logs/event-api.md#emit-event): - The event name (the value of the `event.name` attribute) - - The event domain (the value of the `event.domain` attribute) Things not listed in the above are not expected to remain stable via semantic convention and are allowed (or expected) to change. A few examples: