diff --git a/docs/README.md b/docs/README.md index c03042a..eaed9c9 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,13 +11,22 @@ scheme that can be standardized across a codebase, libraries, and platforms. This allows easier correlation and consumption of data. Contrast standardizes on otel metrics and extend from them. See -[OTel Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/tree/v1.22.0/docs) -for a foundational understanding of what we build on top of. +[OTel Semantic Conventions v1.22.0](https://github.com/open-telemetry/semantic-conventions/tree/v1.22.0/docs) +for a foundational understanding of what we build on top of. As newer versions of the OTel Semantic Conventions are released +this document will be updated as needed to try and conform for maximum portability and usability of the data our sensors +produce. -Security Observability Semantic Conventions are defined for the following areas: - -- [Actions](actions/README.md): Semantic Conventions for actions. +While the OTel Semantic Conventions encompass a lot of technologies, and is growing, we will try to capture and replicate from +the specification what is important for Contrast's sensors. Semantic Conventions by signals: - [Resource](resource/README.md): Semantic Conventions for resources. +- [Trace](general/trace.md): Semantic Conventions for traces and spans. +- [Metrics](general/metrics.md): Semantic Conventions for metrics. + +Semantic Conventions are defined for the following areas: + +- [General](general/README.md): General Semantic Conventions. +- [Actions](actions/README.md): Semantic Conventions for Contrast Actions. +- [HTTP](http/README.md): Semantic Conventions for HTTP client and server operations. diff --git a/docs/actions/README.md b/docs/actions/README.md index 5e27166..243b29a 100644 --- a/docs/actions/README.md +++ b/docs/actions/README.md @@ -3,47 +3,26 @@ Actions are a concept for Security Observability. They are modeled as a metric so that actions are seen for every requests and collected in a scalable manner for the agent. We will never miss an action on a particular execution path. -The data used in the action is captured as attributes within a span. Since -capturing and processing spans is considered an expensive activity, this data is captured -as part of a sampling activity. + +The data used in the action is captured as attributes within a span of a trace. Since capturing and processing spans is considered an expensive activity, this data is captured as part of a sampling activity. + +Information in traces allow us to construct an action graph of the execution ordering and also gives us the same data used within an action. However, since they are sampled, its possible to miss some execution paths that execute other actions. Metrics contain what actions have occurred on an endpoint and are captured for every request, thus they will never miss a particular action. This fidelity comes at a cost however in that metrics will not contain data used in an +action nor will it contain enough information to determine the action execution order. - [Actions](#actions) - * [Metric: `http.server.action.total`](#metric-httpserveractiontotal) + * [Metrics](#metrics) * [Spans](#spans) ## Actions -### Metric: `http.server.action.total` - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| [`action`](action-spans.md) | string | The type of action that was observed. | `file-open-create`; `authn-request` | Required | -| `http.method` | string | http method used when the action was encountered. | `GET`; `POST` | Required | -| `http.route` | string | http route used when the action was encountered. | `/foo/bar` | Required | - -`action` MUST be one of the following: - -| Value | Description | -|---|---| -| `storage-query` | Functions that execute queries | -| `file-open-create` | file open or create action | -| `url-forward` | Any function designed to forward a request to another URL | -| `url-redirect` | Function that result in an http 302 redirect code sent to the client | -| `host-cmd-exec` | system shell command execution | -| `ldap-query` | Functions that result in and ldap query operation | -| `smtp-exec` | Functions that result in an SMTP command execution | -| `outbound-service-call` | Functions that result in external calls to other services | -| `authn-request` | Functions that perform authentication actions | -| `authz-request` | Functions that perform authorization actions | -| `el-execution` | Spring expression language execution | -| `ognl-execution` | Object-Graph Navigation Language expression execution. | - +### Metrics + +- [Action Metrics](action-metrics.md): Semantic Conventions for Action metrics. ### Spans -- [Action Spans](action-spans.md): Semantic Conventions for Action _spans_. +- [Action Spans](action-spans.md): Semantic Conventions for Action spans. diff --git a/docs/actions/action-metrics.md b/docs/actions/action-metrics.md new file mode 100644 index 0000000..6d85abb --- /dev/null +++ b/docs/actions/action-metrics.md @@ -0,0 +1,176 @@ +# Semantic Conventions for Action Metrics + +Actions occur within the context of an Web Application. + +## HTTP Server + +Applications that are served from an HTTP server. + +### Metric: `http.server.action.total` + +This metric is required. + + +| Name | Instrument Type | Unit (UCUM) | Description | +| -------- | --------------- | ----------- | -------------- | +| `http.server.action.total` | Counter | `{action}` | A counter of actions for contrast | + + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `action` | string | The type of action that was observed. | `file-open-create`; `authn-request` | Required | +| `http.request.method` | string | HTTP request method. [1] | `GET`; `POST`; `HEAD` | Recommended | +| `http.route` | string | The matched route (path template in the format used by the respective server framework). See note below [2] | `/users/:userID?`; `{controller}/{action}/{id?}` | Recommended | + +**[1]:** HTTP request method value SHOULD be "known" to the instrumentation. +By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) +and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + +If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + +If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override +the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named +OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods +(this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + +HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. +Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. +Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + +**[2]:** MUST NOT be populated when this is not supported by the HTTP server framework as the route attribute should have low-cardinality and the URI path can NOT substitute it. +SHOULD include the [application root](/docs/http/http-spans.md#http-server-definitions) if there is one. + +`action` MUST be one of the following: + +| Value | Description | +|---|---| +| `storage-query` | Functions that execute queries | +| `file-open-create` | file open or create action | +| `url-forward` | Any function designed to forward a request to another URL | +| `url-redirect` | Function that result in an http 302 redirect code sent to the client | +| `host-cmd-exec` | system shell command execution | +| `ldap-query` | Functions that result in and ldap query operation | +| `smtp-exec` | Functions that result in an SMTP command execution | +| `outbound-service-call` | Functions that result in external calls to other services | +| `authn-request` | Functions that perform authentication actions | +| `authz-request` | Functions that perform authorization actions | +| `el-execution` | Spring expression language execution | +| `ognl-execution` | Object-Graph Navigation Language expression execution. | + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | + + +### Metric: `http.server.request.duration` + +This metric is required. + +The documentation for this metric is duplicated from the OTEL Semantic Convention +[http metrics](https://github.com/open-telemetry/semantic-conventions/blob/v1.22.0/docs/http/http-metrics.md#metric-httpserverrequestduration) document. +It is replicated here for convenience to the readers and to note that it is a required metric for agents to capture. + +This metric SHOULD be specified with +[`ExplicitBucketBoundaries`](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/metrics/api.md#instrument-advice) +of `[ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10 ]`. + + +| Name | Instrument Type | Unit (UCUM) | Description | +| -------- | --------------- | ----------- | -------------- | +| `http.server.request.duration` | Histogram | `s` | Duration of HTTP server requests. | + + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `error.type` | string | Describes a class of error the operation ended with. [1] | `timeout`; `name_resolution_error`; `500` | Conditionally Required: If request has ended with an error. | +| `http.request.method` | string | HTTP request method. [2] | `GET`; `POST`; `HEAD` | Required | +| `http.response.status_code` | int | [HTTP response status code](https://tools.ietf.org/html/rfc7231#section-6). | `200` | Conditionally Required: If and only if one was received/sent. | +| `http.route` | string | The matched route (path template in the format used by the respective server framework). See note below [3] | `/users/:userID?`; `{controller}/{action}/{id?}` | Conditionally Required: If and only if it's available | +| [`network.protocol.name`](../general/attributes.md) | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [4] | `http`; `spdy` | Recommended: if not default (`http`). | +| [`network.protocol.version`](../general/attributes.md) | string | Version of the protocol specified in `network.protocol.name`. [5] | `1.0`; `1.1`; `2`; `3` | Recommended | +| [`server.address`](../general/attributes.md) | string | Name of the local HTTP server that received the request. [6] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Opt-In | +| [`server.port`](../general/attributes.md) | int | Port of the local HTTP server that received the request. [7] | `80`; `8080`; `443` | Opt-In | +| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `http`; `https` | Required | + +**[1]:** If the request fails with an error before response status code was sent or received, +`error.type` SHOULD be set to exception type or a component-specific low cardinality error code. + +If response status code was sent or received and status indicates an error according to [HTTP span status definition](/docs/http/http-spans.md), +`error.type` SHOULD be set to the status code number (represented as a string), an exception type (if thrown) or a component-specific error code. + +The `error.type` value SHOULD be predictable and SHOULD have low cardinality. +Instrumentations SHOULD document the list of errors they report. + +The cardinality of `error.type` within one instrumentation library SHOULD be low, but +telemetry consumers that aggregate data from multiple instrumentation libraries and applications +should be prepared for `error.type` to have high cardinality at query time, when no +additional filters are applied. + +If the request has completed successfully, instrumentations SHOULD NOT set `error.type`. + +**[2]:** HTTP request method value SHOULD be "known" to the instrumentation. +By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) +and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + +If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + +If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override +the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named +OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods +(this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + +HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. +Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. +Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + +**[3]:** MUST NOT be populated when this is not supported by the HTTP server framework as the route attribute should have low-cardinality and the URI path can NOT substitute it. +SHOULD include the [application root](/docs/http/http-spans.md#http-server-definitions) if there is one. + +**[4]:** The value SHOULD be normalized to lowercase. + +**[5]:** `network.protocol.version` refers to the version of the protocol used and might be different from the protocol client's version. If the HTTP client used has a version of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. + +**[6]:** See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). +> **Warning** +> Since this attribute is based on HTTP headers, opting in to it may allow an attacker +> to trigger cardinality limits, degrading the usefulness of the metric. + +**[7]:** See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). +> **Warning** +> Since this attribute is based on HTTP headers, opting in to it may allow an attacker +> to trigger cardinality limits, degrading the usefulness of the metric. + +`error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `_OTHER` | A fallback error value to be used when the instrumentation does not define a custom value for it. | + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | + diff --git a/docs/actions/action-spans.md b/docs/actions/action-spans.md index 6f704b7..7ebd999 100644 --- a/docs/actions/action-spans.md +++ b/docs/actions/action-spans.md @@ -4,8 +4,7 @@ - [Definitions](#definitions) - [Action Span Attributes](#action-span-attributes) - * [Authentication Span](#authentication-span) - * [Authorization Span](#authorization-span) + * [Java Specific action types](#java-specific-action-types) @@ -17,31 +16,6 @@ The span name MUST be set to the action name: ``` -Valid action names are are listed in the action attribute for metrics: - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `action` | string | The type of action that was observed. | `file-open-create`; `authn-request` | Required | - -`action` MUST be one of the following: - -| Value | Description | -|---|---| -| `storage-query` | Functions that execute queries | -| `file-open-create` | file open or create action | -| `url-forward` | Any function designed to forward a request to another URL | -| `url-redirect` | Function that result in an http 302 redirect code sent to the client | -| `host-cmd-exec` | system shell command execution | -| `ldap-query` | Functions that result in and ldap query operation | -| `smtp-exec` | Functions that result in an SMTP command execution | -| `outbound-service-call` | Functions that result in external calls to other services | -| `authn-request` | Functions that perform authentication actions | -| `authz-request` | Functions that perform authorization actions | -| `el-execution` | Spring expression language execution | -| `ognl-execution` | Object-Graph Navigation Language expression execution. | - - ## Action Span Attributes Each Action Span has attributes that describe the action observed. For instance, an `authn-request` span @@ -49,57 +23,18 @@ will list the authentication mechanism along with other authentication-related a An action span for the `storage-query` action will have a different set of attributes targeted toward describing that action. -### Authentication Span - -Used when span name is `authn-request` - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `contrast.authentication.mechanism` | string | An authentication mechanism is a specific method or approach used to verify the identity of a user, system, or entity attempting to access a resource. | `password`; `token`; `biometric` | Recommended | -| `contrast.authentication.protocol` | string | An authentication protocol is a set of rules and procedures that dictate how authentication mechanisms should operate to establish trust and verify identities securely. | `oauth`; `saml`; `ldap`; `custom` | Recommended | - -`contrast.authentication.mechanism` MUST be one of the following: - -| Value | Description | -|---|---| -| `password` | Users provide a username and password. | -| `certificate` | x509 certificate authentication or similar | -| `token` | Involves using a physical or virtual token to authenticate a user | -| `biometric` | file open or create action | -| `mfa` | Two or more of the above mechanisms are used | - -`contrast.authentication.protocol` MUST be one of the following: - -| Value | Description | -|---|---| -| `saml` | Security Assertion Markup Language | -| `oauth` | Open Authentication and OIDC | -| `ldap` | Lightweight Directory Access Protocol | -| `kerberos` | | - - -### Authorization Span - -Used when span name is `authz-request` - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `contrast.authorization.dac.permission` | string | Permission requested for access to the resource. The values here are very domain specific, but will always be normalized to a lowercase value in the data here. | `read`; `write`; `append`; `delete` | Recommended: If mechanism is 'dac' | -| `contrast.authorization.mac.labels` | string | Labels on the requested resource. The values here are very domain specific, but will always be normalized to a lowercase value in the data here. | `top_secret`; `confidential`; `internal`; `public` | Recommended: If mechanism is 'mac' | -| `contrast.authorization.mechanism` | string | How are authz decisions made for the resource. | `rbac`; `dac`; `pbac` | Recommended | -| `contrast.authorization.rbac.role` | string | Role Requested for authz check. The values here are very domain specific, but will always be normalized to a lowercase value in the data here. | `user`; `editor`; `manager` | Recommended: If mechanism is 'rbac' | - -`contrast.authorization.mechanism` MUST be one of the following: - -| Value | Description | -|---|---| -| `rbac` | Role Based Access Control | -| `abac` | Attribute Based Access Control | -| `mac` | Mandatory Access Control (MAC) is a security model where access to resources is determined by the security labels assigned to subjects (users or processes) and objects (resources). | -| `dac` | Discretionary Access Control (DAC) is a model where owners of resources have the discretion to control access to their resources. | -| `pbac` | Policy Based Access Control | -| `hbac` | History Based Access Control | -| `tbac` | Time Based Access Control | - +* [authn-request](authn-request.md) +* [authz-request](authz-request.md) +* [storage-query](storage-query.md) +* [file-open-create](file-open-create.md) +* [outbound-service-call](outbound-service-call.md) +* [url-forward](url-forward.md) +* [url-redirect](url-redirect.md) +* [host-cmd-exec](host-cmd-exec.md) +* [ldap-query](ldap-query.md) +* [smtp-exec](smtp-exec.md) + +### Java Specific action types + +* [el-execution](el-execution.md) +* [ognl-execution](ognl-execution.md) diff --git a/docs/actions/authn-request.md b/docs/actions/authn-request.md new file mode 100644 index 0000000..e53802a --- /dev/null +++ b/docs/actions/authn-request.md @@ -0,0 +1,29 @@ +# Authentication Requests Semantic Conventions + +Used when span name is `authn-request` + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `contrast.authentication.mechanism` | string | An authentication mechanism is a specific method or approach used to verify the identity of a user, system, or entity attempting to access a resource. | `password`; `token`; `biometric` | Recommended | +| `contrast.authentication.protocol` | string | An authentication protocol is a set of rules and procedures that dictate how authentication mechanisms should operate to establish trust and verify identities securely. | `oauth`; `saml`; `ldap`; `custom` | Recommended | + +`contrast.authentication.mechanism` MUST be one of the following: + +| Value | Description | +|---|---| +| `password` | Users provide a username and password. | +| `certificate` | x509 certificate authentication or similar | +| `token` | Involves using a physical or virtual token to authenticate a user | +| `biometric` | file open or create action | +| `mfa` | Two or more of the above mechanisms are used | + +`contrast.authentication.protocol` MUST be one of the following: + +| Value | Description | +|---|---| +| `saml` | Security Assertion Markup Language | +| `oauth` | Open Authentication and OIDC | +| `ldap` | Lightweight Directory Access Protocol | +| `kerberos` | | + diff --git a/docs/actions/authz-request.md b/docs/actions/authz-request.md new file mode 100644 index 0000000..7bc55cf --- /dev/null +++ b/docs/actions/authz-request.md @@ -0,0 +1,24 @@ +# Authorization Span Semantic Convention + +Used when span name is `authz-request` + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `contrast.authorization.dac.permission` | string | Permission requested for access to the resource. The values here are very domain specific, but will always be normalized to a lowercase value in the data here. | `read`; `write`; `append`; `delete` | Recommended: If mechanism is 'dac' | +| `contrast.authorization.mac.labels` | string | Labels on the requested resource. The values here are very domain specific, but will always be normalized to a lowercase value in the data here. | `top_secret`; `confidential`; `internal`; `public` | Recommended: If mechanism is 'mac' | +| `contrast.authorization.mechanism` | string | How are authz decisions made for the resource. | `rbac`; `dac`; `pbac` | Recommended | +| `contrast.authorization.rbac.role` | string | Role Requested for authz check. The values here are very domain specific, but will always be normalized to a lowercase value in the data here. | `user`; `editor`; `manager` | Recommended: If mechanism is 'rbac' | + +`contrast.authorization.mechanism` MUST be one of the following: + +| Value | Description | +|---|---| +| `rbac` | Role Based Access Control | +| `abac` | Attribute Based Access Control | +| `mac` | Mandatory Access Control (MAC) is a security model where access to resources is determined by the security labels assigned to subjects (users or processes) and objects (resources). | +| `dac` | Discretionary Access Control (DAC) is a model where owners of resources have the discretion to control access to their resources. | +| `pbac` | Policy Based Access Control | +| `hbac` | History Based Access Control | +| `tbac` | Time Based Access Control | + diff --git a/docs/actions/el-execution.md b/docs/actions/el-execution.md new file mode 100644 index 0000000..5c2a50b --- /dev/null +++ b/docs/actions/el-execution.md @@ -0,0 +1,9 @@ +# Semantic Conventions for `el-execution` Actions + +Used when span name is `el-execution`. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `code.contents` | string | The code representing the expression being executed. | `#{'String1 ' + 'string2'}`; `#{20 - 1}`; `'Just a string value'.substring(5)` | Required | + diff --git a/docs/actions/file-open-create.md b/docs/actions/file-open-create.md new file mode 100644 index 0000000..295ce3b --- /dev/null +++ b/docs/actions/file-open-create.md @@ -0,0 +1,18 @@ +# Semantic Conventions for `file-open-create` Actions + +Used when span name is `file-open-create`. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `file.open.flags` | string | The flags used when the file was opened or created. | `o_rdonly`; `o_rdwr` | Recommended | +| `file.open.path` | string | The absolute path that was accessed. | `/etc/myconfig`; `/foo/bar`; `/some/tmp` | Required | + +`file.open.flags` MUST be one of the following: + +| Value | Description | +|---|---| +| `o_rdonly` | Read only access | +| `o_wronly` | Write only access | +| `o_rdwr` | Read/write access | + diff --git a/docs/actions/host-cmd-exec.md b/docs/actions/host-cmd-exec.md new file mode 100644 index 0000000..bc5b29a --- /dev/null +++ b/docs/actions/host-cmd-exec.md @@ -0,0 +1,9 @@ +# Semantic Conventions for `host-cmd-exec` Actions + +Used when span name is `host-cmd-exec`. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `cmd` | string | String of executed command with its arguments. | `ls /foo`; `bash -c somebin`; `chmod 755 foobar` | Required | + diff --git a/docs/actions/ldap-query.md b/docs/actions/ldap-query.md new file mode 100644 index 0000000..541571e --- /dev/null +++ b/docs/actions/ldap-query.md @@ -0,0 +1,3 @@ +# Semantic Conventions for `ldap-query` Actions + +Used when span name is `ldap-query`. **TBD** diff --git a/docs/actions/ognl-execution.md b/docs/actions/ognl-execution.md new file mode 100644 index 0000000..c331db8 --- /dev/null +++ b/docs/actions/ognl-execution.md @@ -0,0 +1,9 @@ +# Semantic Conventions for `ognl-execution` Actions + +Used when span name is `ognl-execution`. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `code.contents` | string | The code representing the expression being executed. | `#{'String1 ' + 'string2'}`; `#{20 - 1}`; `'Just a string value'.substring(5)` | Required | + diff --git a/docs/actions/outbound-service-call.md b/docs/actions/outbound-service-call.md new file mode 100644 index 0000000..103e04f --- /dev/null +++ b/docs/actions/outbound-service-call.md @@ -0,0 +1,82 @@ +# Semantic Conventions for `outbound-service-call` Actions + + + + + +- [outbound-service-call (http client) span duration](#outbound-service-call-http-client-span-duration) +- [HTTP request retries and redirects](#http-request-retries-and-redirects) + + + +Used when span name is `outbound-service-call`. + +This span type represents an outbound HTTP request. There are two ways this can be achieved in an instrumentation: + +1. Instrumentations SHOULD create an `outbound-service-call` span for each attempt to send an HTTP request over the wire. + In case the request is resent, the resend attempts MUST follow the [HTTP resend spec](#http-request-retries-and-redirects). + In this case, instrumentations SHOULD NOT (also) emit a logical encompassing `outbound-service-call` span. + +2. If for some reason it is not possible to emit a span for each send attempt (because e.g. the instrumented library does not expose hooks that would allow this), + instrumentations MAY create an `outbound-service-call` span for the top-most operation of the HTTP client. + In this case, the `url.full` MUST be the absolute URL that was originally requested, before any HTTP-redirects that may happen when executing the request. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `http.resend_count` | int | The ordinal number of request resending attempt (for any reason, including redirects). [1] | `3` | Recommended: if and only if request was retried. | +| [`network.peer.address`](../general/attributes.md) | string | Peer address of the network connection - IP address or Unix domain socket name. | `10.1.2.80`; `/tmp/my.sock` | Recommended: If different than `server.address`. | +| [`network.peer.port`](../general/attributes.md) | int | Peer port number of the network connection. | `65123` | Recommended: If `network.peer.address` is set. | +| [`server.address`](../general/attributes.md) | string | Host identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [2] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Required | +| [`server.port`](../general/attributes.md) | int | Port identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [3] | `80`; `8080`; `443` | Conditionally Required: [4] | +| `url.full` | string | Absolute URL describing a network resource according to [RFC3986](https://www.rfc-editor.org/rfc/rfc3986) [5] | `https://www.foo.bar/search?q=OpenTelemetry#SemConv`; `//localhost` | Required | + +**[1]:** The resend count SHOULD be updated each time an HTTP request gets resent by the client, regardless of what was the cause of the resending (e.g. redirection, authorization failure, 503 Server Unavailable, network issues, or any other). + +**[2]:** Determined by using the first of the following that applies + +- Host identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form +- Host identifier of the `Host` header + +If an HTTP client request is explicitly made to an IP address, e.g. `http://x.x.x.x:8080`, then +`server.address` SHOULD be the IP address `x.x.x.x`. A DNS lookup SHOULD NOT be used. + +**[3]:** When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `server.port` MUST match URI port identifier, otherwise it MUST match `Host` header port identifier. + +**[4]:** If not default (`80` for `http` scheme, `443` for `https`). + +**[5]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless. +`url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`. +`url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) and SHOULD NOT be validated or modified except for sanitizing purposes. + +Following attributes MUST be provided **at span creation time** (when provided at all), so they can be considered for sampling decisions: + +* [`server.address`](../general/attributes.md) +* [`server.port`](../general/attributes.md) +* `url.full` + + +## outbound-service-call (http client) span duration + +There are some minimal constraints that SHOULD be honored: + +* `outbound-service-call` spans SHOULD start sometime before the first request byte is sent. This may or may not include connection time. +* `outbound-service-call` spans SHOULD end sometime after the HTTP response headers are fully read (or when they fail to be read). This may or may not include reading the response body. + +If there is any possibility for application code to not fully read the HTTP response +(and for the HTTP client library to then have to clean up the HTTP response asynchronously), +the HTTP client span SHOULD NOT be ended in this cleanup phase, +and instead SHOULD end at some point after the HTTP response headers are fully read (or fail to be read). +This avoids the span being ended asynchronously later on at a time +which is no longer directly associated with the application code which made the HTTP request. + +Because of the potential for confusion around this, HTTP client library instrumentations SHOULD document their behavior around ending HTTP client spans. + +## HTTP request retries and redirects + +Retries and redirects cause more than one physical HTTP request to be sent. +A request is resent when an HTTP client library sends more than one HTTP request to satisfy the same API call. +This may happen due to following redirects, authorization challenges, 503 Server Unavailable, network issues, or any other. + +Each time an HTTP request is resent, the `http.resend_count` attribute SHOULD be added to each repeated span and set to the ordinal number of the request resend attempt. diff --git a/docs/actions/smtp-exec.md b/docs/actions/smtp-exec.md new file mode 100644 index 0000000..5081a54 --- /dev/null +++ b/docs/actions/smtp-exec.md @@ -0,0 +1,3 @@ +# Semantic Conventions for `smtp-exec` Actions + +Used when span name is `smtp-exec`. **TBD** diff --git a/docs/actions/storage-query.md b/docs/actions/storage-query.md new file mode 100644 index 0000000..1cdcb70 --- /dev/null +++ b/docs/actions/storage-query.md @@ -0,0 +1,7 @@ +# Semantic Conventions for `storage-query` Actions + +Used when span name is `storage-query` + +The attributes used in this span follows the OTEL Database Span semantic +conventions without any modification. They can be found here: +[OTEL Database Span Sematic Conventions](https://github.com/open-telemetry/semantic-conventions/blob/v1.22.0/docs/database/database-spans.md) diff --git a/docs/actions/url-forward.md b/docs/actions/url-forward.md new file mode 100644 index 0000000..504e444 --- /dev/null +++ b/docs/actions/url-forward.md @@ -0,0 +1,3 @@ +# Semantic Conventions for `url-forward` Actions + +Used when span name is `url-forward`. **TBD** diff --git a/docs/actions/url-redirect.md b/docs/actions/url-redirect.md new file mode 100644 index 0000000..3ba1015 --- /dev/null +++ b/docs/actions/url-redirect.md @@ -0,0 +1,3 @@ +# Semantic Conventions for `url-redirect` Actions + +Used when span name is `url-redirect`. **TBD** diff --git a/docs/general/README.md b/docs/general/README.md new file mode 100644 index 0000000..b7b1b46 --- /dev/null +++ b/docs/general/README.md @@ -0,0 +1,9 @@ +# General Semantic Conventions + +This document defines general Semantic Conventions for spans and metrics. + +The following general Semantic Conventions are defined: + +* **[General attributes](attributes.md): General semantic attributes**. +* [Metrics](metrics.md): General Semantic Conventions for metrics. +* [Spans](trace.md): General Semantic Conventions for traces / spans. diff --git a/docs/general/attributes.md b/docs/general/attributes.md new file mode 100644 index 0000000..b1aaa86 --- /dev/null +++ b/docs/general/attributes.md @@ -0,0 +1,254 @@ +# General Attributes + +The attributes described in this section are not specific to a particular operation but rather generic. +They may be used in any Span they apply to. +Particular operations may refer to or require some of these attributes. + + + + + +- [Server, client and shared network attributes](#server-client-and-shared-network-attributes) + * [Address and port attributes](#address-and-port-attributes) + * [Server attributes](#server-attributes) + + [`server.address`](#serveraddress) + * [Client attributes](#client-attributes) + * [Source and destination attributes](#source-and-destination-attributes) + + [Source](#source) + + [Destination](#destination) + * [Other network attributes](#other-network-attributes) + + [`network.peer.*` and `network.local.*` attributes](#networkpeer-and-networklocal-attributes) + - [Client/server examples using `network.peer.*`](#clientserver-examples-using--networkpeer) + * [Simple client/server example](#simple-clientserver-example) + * [Client/server example with reverse proxy](#clientserver-example-with-reverse-proxy) + * [Client/server example with forward proxy](#clientserver-example-with-forward-proxy) +- [General remote service attributes](#general-remote-service-attributes) +- [Source Code Attributes](#source-code-attributes) + + + + + + +## Server, client and shared network attributes + +These attributes may be used to describe the client and server in a connection-based network interaction +where there is one side that initiates the connection (the client is the side that initiates the connection). +This covers all TCP network interactions since TCP is connection-based and one side initiates the +connection (an exception is made for peer-to-peer communication over TCP where the "user-facing" surface of the +protocol / API does not expose a clear notion of client and server). +This also covers UDP network interactions where one side initiates the interaction, e.g. QUIC (HTTP/3) and DNS. + +In an ideal situation, not accounting for proxies, multiple IP addresses or host names, +the `server.*` attributes are the same on the client and server. + +### Address and port attributes + +For all IP-based protocols, the "address" should be just the IP-level address. +Protocol-specific parts of an address are split into other attributes (when applicable) such as "port" attributes for +TCP and UDP. If such transport-specific information is collected and the attribute name does not already uniquely +identify the transport, then setting [`network.transport`](#other-network-attributes) is especially encouraged. + +### Server attributes + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `server.address` | string | Server address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. [1] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Recommended | +| `server.port` | int | Server port number. [2] | `80`; `8080`; `443` | Recommended | + +**[1]:** When observed from the client side, and when communicating through an intermediary, `server.address` SHOULD represent +the server address behind any intermediaries (e.g. proxies) if it's available. + +**[2]:** When observed from the client side, and when communicating through an intermediary, `server.port` SHOULD represent the server port behind any intermediaries (e.g. proxies) if it's available. + + +`server.address` and `server.port` represent logical server name and port. Semantic conventions that refer to these attributes SHOULD +specify what these attributes mean in their context. + +#### `server.address` + +For IP-based communication, the name should be a DNS host name of the service. On client side it matches remote service name, on server side, it represents local service name as seen externally on clients. + +When connecting to an URL `https://example.com/foo`, `server.address` matches `"example.com"` on both client and server side. + +On client side, it's usually passed in form of URL, connection string, host name, etc. Sometimes host name is only available to instrumentation as a string which may contain DNS name or IP address. `server.address` SHOULD be set to the available known hostname (e.g., `"127.0.0.1"` if connecting to an URL `https://127.0.0.1/foo`). + +If only IP address is available, it should be populated on `server.address`. Reverse DNS lookup SHOULD NOT be used to obtain DNS name. + +If `network.transport` is `"pipe"`, the absolute path to the file representing it should be used as `server.address`. +If there is no such file (e.g., anonymous pipe), +the name should explicitly be set to the empty string to distinguish it from the case where the name is just unknown or not covered by the instrumentation. + +For Unix domain socket, `server.address` attribute represents remote endpoint address on the client side and local endpoint address on the server side. + +### Client attributes + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `client.address` | string | Client address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. [1] | `client.example.com`; `10.1.2.80`; `/tmp/my.sock` | Recommended | +| `client.port` | int | Client port number. [2] | `65123` | Recommended | + +**[1]:** When observed from the server side, and when communicating through an intermediary, `client.address` SHOULD represent the client address behind any intermediaries (e.g. proxies) if it's available. + +**[2]:** When observed from the server side, and when communicating through an intermediary, `client.port` SHOULD represent the client port behind any intermediaries (e.g. proxies) if it's available. + + +### Source and destination attributes + +These attributes may be used to describe the sender and receiver of a network exchange/packet. These should be used +when there is no client/server relationship between the two sides, or when that relationship is unknown. +This covers low-level network interactions (e.g. packet tracing) where you don't know if +there was a connection or which side initiated it. +This also covers unidirectional UDP flows and peer-to-peer communication where the +"user-facing" surface of the protocol / API does not expose a clear notion of client and server. + +#### Source + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `source.address` | string | Source address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. [1] | `source.example.com`; `10.1.2.80`; `/tmp/my.sock` | Recommended | +| `source.port` | int | Source port number | `3389`; `2888` | Recommended | + +**[1]:** When observed from the destination side, and when communicating through an intermediary, `source.address` SHOULD represent the source address behind any intermediaries (e.g. proxies) if it's available. + + +#### Destination + +Destination fields capture details about the receiver of a network exchange/packet. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `destination.address` | string | Destination address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. [1] | `destination.example.com`; `10.1.2.80`; `/tmp/my.sock` | Recommended | +| `destination.port` | int | Destination port number | `3389`; `2888` | Recommended | + +**[1]:** When observed from the source side, and when communicating through an intermediary, `destination.address` SHOULD represent the destination address behind any intermediaries (e.g. proxies) if it's available. + + + + +### Other network attributes + +> **Warning** +> Attributes in this section are in use by the HTTP semantic conventions. +Once the HTTP semantic conventions are declared stable, changes to the attributes in this section will only be allowed +if they do not cause breaking changes to HTTP semantic conventions. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `network.local.address` | string | Local address of the network connection - IP address or Unix domain socket name. | `10.1.2.80`; `/tmp/my.sock` | Recommended | +| `network.local.port` | int | Local port number of the network connection. | `65123` | Recommended | +| `network.peer.address` | string | Peer address of the network connection - IP address or Unix domain socket name. | `10.1.2.80`; `/tmp/my.sock` | Recommended | +| `network.peer.port` | int | Peer port number of the network connection. | `65123` | Recommended | +| `network.protocol.name` | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [1] | `amqp`; `http`; `mqtt` | Recommended | +| `network.protocol.version` | string | Version of the protocol specified in `network.protocol.name`. [2] | `3.1.1` | Recommended | +| `network.transport` | string | [OSI transport layer](https://osi-model.com/transport-layer/) or [inter-process communication method](https://en.wikipedia.org/wiki/Inter-process_communication). [3] | `tcp`; `udp` | Recommended | +| `network.type` | string | [OSI network layer](https://osi-model.com/network-layer/) or non-OSI equivalent. [4] | `ipv4`; `ipv6` | Recommended | + +**[1]:** The value SHOULD be normalized to lowercase. + +**[2]:** `network.protocol.version` refers to the version of the protocol used and might be different from the protocol client's version. If the HTTP client used has a version of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. + +**[3]:** The value SHOULD be normalized to lowercase. + +Consider always setting the transport when setting a port number, since +a port number is ambiguous without knowing the transport, for example +different processes could be listening on TCP port 12345 and UDP port 12345. + +**[4]:** The value SHOULD be normalized to lowercase. + +`network.transport` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `tcp` | TCP | +| `udp` | UDP | +| `pipe` | Named or anonymous pipe. See note below. | +| `unix` | Unix domain socket | + +`network.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `ipv4` | IPv4 | +| `ipv6` | IPv6 | + + +#### `network.peer.*` and `network.local.*` attributes + +These attributes identify network peers that are directly connected to each other. + +`network.peer.address` and `network.local.address` should be IP addresses, Unix domain socket names, or other addresses specific to network type. + +_Note: Specific structures and methods to obtain socket-level attributes are mentioned here only as examples. Instrumentations would usually use Socket API provided by their environment or sockets implementations._ + +When connecting using `connect(2)` ([Linux or other POSIX systems](https://man7.org/linux/man-pages/man2/connect.2.html) / +[Windows](https://docs.microsoft.com/windows/win32/api/winsock2/nf-winsock2-connect)) +or `bind(2)`([Linux or other POSIX systems](https://man7.org/linux/man-pages/man2/bind.2.html) / +[Windows](https://docs.microsoft.com/windows/win32/api/winsock2/nf-winsock2-bind)) +with `AF_INET` address family, `network.peer.address` and `network.peer.port` represent `sin_addr` and `sin_port` fields +of `sockaddr_in` structure. + +`network.peer.address` and `network.peer.port` can be obtained by calling `getpeername` method +([Linux or other POSIX systems](https://man7.org/linux/man-pages/man2/getpeername.2.html) / +[Windows](https://docs.microsoft.com/windows/win32/api/winsock2/nf-winsock2-getpeername)). + +`network.local.address` and `network.local.port` can be obtained by calling `getsockname` method +([Linux or other POSIX systems](https://man7.org/linux/man-pages/man2/getsockname.2.html) / +[Windows](https://docs.microsoft.com/windows/win32/api/winsock2/nf-winsock2-getsockname)). + +##### Client/server examples using `network.peer.*` + +Note that `network.local.*` attributes are not included in these examples since they are typically Opt-In. + +###### Simple client/server example + +![simple.png](simple.png) + +###### Client/server example with reverse proxy + +![reverse-proxy.png](reverse-proxy.png) + +###### Client/server example with forward proxy + +![forward-proxy.png](forward-proxy.png) + +## General remote service attributes + +This attribute may be used for any operation that accesses some remote service. +Users can define what the name of a service is based on their particular semantics in their distributed system. +Instrumentations SHOULD provide a way for users to configure this name. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `peer.service` | string | The [`service.name`](/docs/resource/README.md#service) of the remote service. SHOULD be equal to the actual `service.name` resource attribute of the remote service if any. | `AuthTokenCache` | Recommended | + + +Examples of `peer.service` that users may specify: + +- A Redis cache of auth tokens as `peer.service="AuthTokenCache"`. +- A gRPC service `rpc.service="io.opentelemetry.AuthService"` may be hosted in both a gateway, `peer.service="ExternalApiService"` and a backend, `peer.service="AuthService"`. + +## Source Code Attributes + +Often a span is closely tied to a certain unit of code that is logically responsible for handling +the operation that the span describes (usually the method that starts the span). +For an HTTP server span, this would be the function that handles the incoming request, for example. +The attributes listed below allow to report this unit of code and therefore to provide more context +about the span. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `code.column` | int | The column number in `code.filepath` best representing the operation. It SHOULD point within the code unit named in `code.function`. | `16` | Recommended | +| `code.filepath` | string | The source code file name that identifies the code unit as uniquely as possible (preferably an absolute file path). | `/usr/local/MyApplication/content_root/app/index.php` | Recommended | +| `code.function` | string | The method or function name, or equivalent (usually rightmost part of the code unit's name). | `serveRequest` | Recommended | +| `code.lineno` | int | The line number in `code.filepath` best representing the operation. It SHOULD point within the code unit named in `code.function`. | `42` | Recommended | +| `code.namespace` | string | The "namespace" within which `code.function` is defined. Usually the qualified class or module name, such that `code.namespace` + some separator + `code.function` form a unique identifier for the code unit. | `com.example.MyHttpService` | Recommended | +| `code.stacktrace` | string | A stacktrace as a string in the natural representation for the language runtime. The representation is to be determined and documented by each language SIG. | `at com.example.GenerateTrace.methodB(GenerateTrace.java:13)\n at com.example.GenerateTrace.methodA(GenerateTrace.java:9)\n at com.example.GenerateTrace.main(GenerateTrace.java:5)` | Opt-In | + diff --git a/docs/general/forward-proxy.png b/docs/general/forward-proxy.png new file mode 100644 index 0000000..0badc46 Binary files /dev/null and b/docs/general/forward-proxy.png differ diff --git a/docs/general/metrics.md b/docs/general/metrics.md new file mode 100644 index 0000000..5bacfa7 --- /dev/null +++ b/docs/general/metrics.md @@ -0,0 +1,213 @@ +# Metrics Semantic Conventions + + + +- [General Guidelines](#general-guidelines) + * [Name Reuse Prohibition](#name-reuse-prohibition) + * [Units](#units) + * [Naming rules for Counters and UpDownCounters](#naming-rules-for-counters-and-updowncounters) + + [Pluralization](#pluralization) + + [Use `count` Instead of Pluralization for UpDownCounters](#use-count-instead-of-pluralization-for-updowncounters) + + [Do not use `total`](#do-not-use-total) +- [General Metric Semantic Conventions](#general-metric-semantic-conventions) + * [Instrument Naming](#instrument-naming) + * [Instrument Units](#instrument-units) + * [Instrument Types](#instrument-types) + * [Consistent UpDownCounter timeseries](#consistent-updowncounter-timeseries) + + + +The following semantic conventions surrounding metrics are defined: + +* **[General Guidelines](#general-guidelines): General metrics guidelines.** +* [Actions](../actions/action-metrics.md): For Contrast Action metrics. +* [HTTP](../http/http-metrics.md): For HTTP client and server metrics. + +Apart from semantic conventions for metrics, [traces](trace.md), OpenTelemetry also +defines the concept of overarching [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/resource/sdk.md) with +their own [Resource Semantic Conventions](../resource/README.md). + +## General Guidelines + +Metric names and attributes exist within a single universe and a single +hierarchy. Metric names and attributes MUST be considered within the universe of +all existing metric names. When defining new metric names and attributes, +consider the prior art of existing standard metrics and metrics from +frameworks/libraries. + +Associated metrics SHOULD be nested together in a hierarchy based on their +usage. Define a top-level hierarchy for common metric categories: for OS +metrics, like CPU and network; for app runtimes, like GC internals. Libraries +and frameworks should nest their metrics into a hierarchy as well. This aids +in discovery and adhoc comparison. This allows a user to find similar metrics +given a certain metric. + +The hierarchical structure of metrics defines the namespacing. Supporting +OpenTelemetry artifacts define the metric structures and hierarchies for some +categories of metrics, and these can assist decisions when creating future +metrics. + +Common attributes SHOULD be consistently named. This aids in discoverability and +disambiguates similar attributes to metric names. + +["As a rule of thumb, **aggregations** over all the attributes of a given +metric **SHOULD** be +meaningful,"](https://prometheus.io/docs/practices/naming/#metric-names) as +Prometheus recommends. + +Semantic ambiguity SHOULD be avoided. Use prefixed metric names in cases +where similar metrics have significantly different implementations across the +breadth of all existing metrics. For example, every garbage collected runtime +has slightly different strategies and measures. Using a single set of metric +names for GC, not divided by the runtime, could create dissimilar comparisons +and confusion for end users. (For example, prefer `process.runtime.java.gc*` over +`process.runtime.gc.*`.) Measures of many operating system metrics are similarly +ambiguous. + +### Name Reuse Prohibition + +A new metric MUST NOT be added with the same name as a metric that existed in +the past but was renamed (with a corresponding schema file). + +When introducing a new metric name check all existing schema files to make sure +the name does not appear as a key of any "rename_metrics" section (keys denote +old metric names in rename operations). + +### Units + +Conventional metrics or metrics that have their units included in +OpenTelemetry metadata (e.g. `metric.WithUnit` in Go) SHOULD NOT include the +units in the metric name. Units may be included when it provides additional +meaning to the metric name. Metrics MUST, above all, be understandable and +usable. + +When building components that interoperate between OpenTelemetry and a system +using the OpenMetrics exposition format, use the +[OpenMetrics Guidelines](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/compatibility/prometheus_and_openmetrics.md). + +### Naming rules for Counters and UpDownCounters + +#### Pluralization + +Metric namespaces SHOULD NOT be pluralized. + +Metric names SHOULD NOT be pluralized, unless the value being recorded +represents discrete instances of a +[countable quantity](https://wikipedia.org/wiki/Count_noun). +Generally, the name SHOULD be pluralized only if the unit of the metric in +question is a non-unit (like `{fault}` or `{operation}`). + +Examples: + +* `system.filesystem.utilization`, `http.server.request.duration`, and `system.cpu.time` +should not be pluralized, even if many data points are recorded. +* `system.paging.faults`, `system.disk.operations`, and `system.network.packets` +should be pluralized, even if only a single data point is recorded. + +#### Use `count` Instead of Pluralization for UpDownCounters + +If the value being recorded represents the count of concepts signified +by the namespace then the metric should be named `count` (within its namespace). + +For example if we have a namespace `system.process` which contains all metrics related +to the processes then to represent the count of the processes we can have a metric named +`system.process.count`. + +#### Do not use `total` + +UpDownCounters SHOULD NOT use `_total` because then they will look like +monotonic sums. + +Counters SHOULD NOT append `_total` either because then their meaning will +be confusing in delta backends. + +## General Metric Semantic Conventions + +The following semantic conventions aim to keep naming consistent. They +provide guidelines for most of the cases in this specification and should be +followed for other instruments not explicitly defined in this document. + +### Instrument Naming + +- **limit** - an instrument that measures the constant, known total amount of +something should be called `entity.limit`. For example, `system.memory.limit` +for the total amount of memory on a system. + +- **usage** - an instrument that measures an amount used out of a known total +(**limit**) amount should be called `entity.usage`. For example, +`system.memory.usage` with attribute `state = used | cached | free | ...` for the +amount of memory in a each state. Where appropriate, the sum of **usage** +over all attribute values SHOULD be equal to the **limit**. + + A measure of the amount consumed of an unlimited resource, or of a resource + whose limit is unknowable, is differentiated from **usage**. For example, the + maximum possible amount of virtual memory that a process may consume may + fluctuate over time and is not typically known. + +- **utilization** - an instrument that measures the *fraction* of **usage** +out of its **limit** should be called `entity.utilization`. For example, +`system.memory.utilization` for the fraction of memory in use. Utilization can +be with respect to a fixed limit or a soft limit. Utilization values are +represended as a ratio and are typically in the range `[0, 1]`, but may go above 1 +in case of exceeding a soft limit. + +- **time** - an instrument that measures passage of time should be called +`entity.time`. For example, `system.cpu.time` with attribute `state = idle | user +| system | ...`. **time** measurements are not necessarily wall time and can +be less than or greater than the real wall time between measurements. + + **time** instruments are a special case of **usage** metrics, where the + **limit** can usually be calculated as the sum of **time** over all attribute + values. **utilization** for time instruments can be derived automatically + using metric event timestamps. For example, `system.cpu.utilization` is + defined as the difference in `system.cpu.time` measurements divided by the + elapsed time and number of CPUs. + +- **io** - an instrument that measures bidirectional data flow should be +called `entity.io` and have attributes for direction. For example, +`system.network.io`. + +- Other instruments that do not fit the above descriptions may be named more +freely. For example, `system.paging.faults` and `system.network.packets`. +Units do not need to be specified in the names since they are included during +instrument creation, but can be added if there is ambiguity. + +### Instrument Units + +Units should follow the +[Unified Code for Units of Measure](http://unitsofmeasure.org/ucum.html). + +- Instruments for **utilization** metrics (that measure the fraction out of a +total) are dimensionless and SHOULD use the default unit `1` (the unity). +- All non-units that use curly braces to annotate a quantity need to match the + grammatical number of the quantity it represent. For example if measuring the + number of individual requests to a process the unit would be `{request}`, not + `{requests}`. +- Instruments that measure an integer count of something SHOULD only use +[annotations](https://ucum.org/ucum.html#para-curly) with curly braces to +give additional meaning *without* the leading default unit (`1`). For example, +use `{packet}`, `{error}`, `{fault}`, etc. +- Instrument units other than `1` and those that use + [annotations](https://ucum.org/ucum.html#para-curly) SHOULD be specified using + the UCUM case sensitive ("c/s") variant. + For example, "Cel" for the unit with full name "degree Celsius". +- Instruments SHOULD use non-prefixed units (i.e. `By` instead of `MiBy`) + unless there is good technical reason to not do so. +- When instruments are measuring durations, seconds (i.e. `s`) SHOULD be used. + +### Instrument Types + +The semantic metric conventions specification is written to use the names of the synchronous instrument types, +like `Counter` or `UpDownCounter`. However, compliant implementations MAY use the asynchronous equivalent instead, +like `Asynchronous Counter` or `Asynchronous UpDownCounter`. +Whether implementations choose the synchronous type or the asynchronous equivalent is considered to be an +implementation detail. Both choices are compliant with this specification. + +### Consistent UpDownCounter timeseries + +When recording `UpDownCounter` metrics, the same attribute values used to record an increment SHOULD be used to record +any associated decrement, otherwise those increments and decrements will end up as different timeseries. + +For example, if you are tracking `active_requests` with an `UpDownCounter`, and you are incrementing it each time a +request starts and decrementing it each time a request ends, then any attributes which are not yet available when +incrementing the counter at request start should not be used when decrementing the counter at request end. diff --git a/docs/general/reverse-proxy.png b/docs/general/reverse-proxy.png new file mode 100644 index 0000000..2f21f81 Binary files /dev/null and b/docs/general/reverse-proxy.png differ diff --git a/docs/general/simple.png b/docs/general/simple.png new file mode 100644 index 0000000..b8c48b0 Binary files /dev/null and b/docs/general/simple.png differ diff --git a/docs/general/trace.md b/docs/general/trace.md new file mode 100644 index 0000000..82671b9 --- /dev/null +++ b/docs/general/trace.md @@ -0,0 +1,14 @@ +# Trace Semantic Convention + +For an encompassing description of trace semantics, see +[OTEL Trace Semantic Convention](https://github.com/open-telemetry/semantic-conventions/blob/v1.22.0/docs/general/trace.md). +The attributes described in this document will only described new attributes +added by Contrast Security or certain required attributes and highly desired +recommended attributes. However, all agents should strive to fill in as much data +as resonable guided by the OTEL specification. + +The following semantic conventions for Contrast Spans are defined: + +- [General](attributes.md): General semantic attributes that may be used in describing different kinds of operations. +- [Actions](../actions/action-spans.md): For spans describing Contrast Actions. +- [HTTP](../http/http-spans.md): For HTTP client and server spans. diff --git a/docs/http/README.md b/docs/http/README.md new file mode 100644 index 0000000..17a8b4f --- /dev/null +++ b/docs/http/README.md @@ -0,0 +1,10 @@ +# Semantic Conventions for HTTP + +This document defines semantic conventions for HTTP spans and metrics. +They can be used for http and https schemes +and various HTTP versions like 1.1, 2 and SPDY. + +Semantic conventions for HTTP are defined for the following signals: + +* [HTTP Spans](http-spans.md): Semantic Conventions for HTTP client and server *spans*. +* [HTTP Metrics](http-metrics.md): Semantic Conventions for HTTP client and server *metrics*. diff --git a/docs/http/http-metrics.md b/docs/http/http-metrics.md new file mode 100644 index 0000000..1e32971 --- /dev/null +++ b/docs/http/http-metrics.md @@ -0,0 +1,483 @@ +# Semantic Conventions for HTTP Metrics + +**Status**: [Mixed][DocumentStatus] + +The conventions described in this section are HTTP specific. When HTTP operations occur, +metric events about those operations will be generated and reported to provide insight into the +operations. By adding HTTP attributes to metric events it allows for finely tuned filtering. + +**Disclaimer:** These are initial HTTP metric instruments and attributes but more may be added in the future. + + + +- [HTTP Server](#http-server) + * [Metric: `http.server.request.duration`](#metric-httpserverrequestduration) + * [Metric: `http.server.active_requests`](#metric-httpserveractive_requests) + * [Metric: `http.server.request.body.size`](#metric-httpserverrequestbodysize) +- [HTTP Client](#http-client) + * [Metric: `http.client.request.duration`](#metric-httpclientrequestduration) + * [Metric: `http.client.request.body.size`](#metric-httpclientrequestbodysize) + * [Metric: `http.client.response.body.size`](#metric-httpclientresponsebodysize) + + + +## HTTP Server + +### Metric: `http.server.request.duration` + +This metric is required. + +When this metric is reported alongside an HTTP server span, the metric value SHOULD be the same as the HTTP server span duration. + +This metric SHOULD be specified with +[`ExplicitBucketBoundaries`](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/metrics/api.md#instrument-advisory-parameters) +of `[ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10 ]`. + + +| Name | Instrument Type | Unit (UCUM) | Description | +| -------- | --------------- | ----------- | -------------- | +| `http.server.request.duration` | Histogram | `s` | Duration of HTTP server requests. | + + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `error.type` | string | Describes a class of error the operation ended with. [1] | `timeout`; `name_resolution_error`; `500` | Conditionally Required: If request has ended with an error. | +| `http.request.method` | string | HTTP request method. [2] | `GET`; `POST`; `HEAD` | Required | +| `http.response.status_code` | int | [HTTP response status code](https://tools.ietf.org/html/rfc7231#section-6). | `200` | Conditionally Required: If and only if one was received/sent. | +| `http.route` | string | The matched route (path template in the format used by the respective server framework). See note below [3] | `/users/:userID?`; `{controller}/{action}/{id?}` | Conditionally Required: If and only if it's available | +| [`network.protocol.name`](../general/attributes.md) | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [4] | `http`; `spdy` | Recommended: if not default (`http`). | +| [`network.protocol.version`](../general/attributes.md) | string | Version of the protocol specified in `network.protocol.name`. [5] | `1.0`; `1.1`; `2`; `3` | Recommended | +| [`server.address`](../general/attributes.md) | string | Name of the local HTTP server that received the request. [6] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Opt-In | +| [`server.port`](../general/attributes.md) | int | Port of the local HTTP server that received the request. [7] | `80`; `8080`; `443` | Opt-In | +| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `http`; `https` | Required | + +**[1]:** If the request fails with an error before response status code was sent or received, +`error.type` SHOULD be set to exception type or a component-specific low cardinality error code. + +If response status code was sent or received and status indicates an error according to [HTTP span status definition](/docs/http/http-spans.md), +`error.type` SHOULD be set to the status code number (represented as a string), an exception type (if thrown) or a component-specific error code. + +The `error.type` value SHOULD be predictable and SHOULD have low cardinality. +Instrumentations SHOULD document the list of errors they report. + +The cardinality of `error.type` within one instrumentation library SHOULD be low, but +telemetry consumers that aggregate data from multiple instrumentation libraries and applications +should be prepared for `error.type` to have high cardinality at query time, when no +additional filters are applied. + +If the request has completed successfully, instrumentations SHOULD NOT set `error.type`. + +**[2]:** HTTP request method value SHOULD be "known" to the instrumentation. +By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) +and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + +If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + +If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override +the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named +OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods +(this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + +HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. +Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. +Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + +**[3]:** MUST NOT be populated when this is not supported by the HTTP server framework as the route attribute should have low-cardinality and the URI path can NOT substitute it. +SHOULD include the [application root](/docs/http/http-spans.md#http-server-definitions) if there is one. + +**[4]:** The value SHOULD be normalized to lowercase. + +**[5]:** `network.protocol.version` refers to the version of the protocol used and might be different from the protocol client's version. If the HTTP client used has a version of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. + +**[6]:** See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). +> **Warning** +> Since this attribute is based on HTTP headers, opting in to it may allow an attacker +> to trigger cardinality limits, degrading the usefulness of the metric. + +**[7]:** See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). +> **Warning** +> Since this attribute is based on HTTP headers, opting in to it may allow an attacker +> to trigger cardinality limits, degrading the usefulness of the metric. + +`error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `_OTHER` | A fallback error value to be used when the instrumentation does not define a custom value for it. | + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | + + +### Metric: `http.server.active_requests` + +This metric is optional. + + +| Name | Instrument Type | Unit (UCUM) | Description | +| -------- | --------------- | ----------- | -------------- | +| `http.server.active_requests` | UpDownCounter | `{request}` | Number of active HTTP server requests. | + + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `http.request.method` | string | HTTP request method. [1] | `GET`; `POST`; `HEAD` | Required | +| [`server.address`](../general/attributes.md) | string | Name of the local HTTP server that received the request. [2] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Opt-In | +| [`server.port`](../general/attributes.md) | int | Port of the local HTTP server that received the request. [3] | `80`; `8080`; `443` | Opt-In | +| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `http`; `https` | Required | + +**[1]:** HTTP request method value SHOULD be "known" to the instrumentation. +By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) +and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + +If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + +If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override +the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named +OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods +(this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + +HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. +Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. +Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + +**[2]:** See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). +> **Warning** +> Since this attribute is based on HTTP headers, opting in to it may allow an attacker +> to trigger cardinality limits, degrading the usefulness of the metric. + +**[3]:** See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). +> **Warning** +> Since this attribute is based on HTTP headers, opting in to it may allow an attacker +> to trigger cardinality limits, degrading the usefulness of the metric. + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | + + +### Metric: `http.server.request.body.size` + +This metric is optional. + + +| Name | Instrument Type | Unit (UCUM) | Description | +| -------- | --------------- | ----------- | -------------- | +| `http.server.request.body.size` | Histogram | `By` | Size of HTTP server request bodies. [1] | + +**[1]:** The size of the request payload body in bytes. This is the number of bytes transferred excluding headers and is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) header. For requests using transport encoding, this should be the compressed size. + + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `error.type` | string | Describes a class of error the operation ended with. [1] | `timeout`; `name_resolution_error`; `500` | Conditionally Required: If request has ended with an error. | +| `http.request.method` | string | HTTP request method. [2] | `GET`; `POST`; `HEAD` | Required | +| `http.response.status_code` | int | [HTTP response status code](https://tools.ietf.org/html/rfc7231#section-6). | `200` | Conditionally Required: If and only if one was received/sent. | +| `http.route` | string | The matched route (path template in the format used by the respective server framework). See note below [3] | `/users/:userID?`; `{controller}/{action}/{id?}` | Conditionally Required: If and only if it's available | +| [`network.protocol.name`](../general/attributes.md) | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [4] | `http`; `spdy` | Recommended: if not default (`http`). | +| [`network.protocol.version`](../general/attributes.md) | string | Version of the protocol specified in `network.protocol.name`. [5] | `1.0`; `1.1`; `2`; `3` | Recommended | +| [`server.address`](../general/attributes.md) | string | Name of the local HTTP server that received the request. [6] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Opt-In | +| [`server.port`](../general/attributes.md) | int | Port of the local HTTP server that received the request. [7] | `80`; `8080`; `443` | Opt-In | +| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `http`; `https` | Required | + +**[1]:** If the request fails with an error before response status code was sent or received, +`error.type` SHOULD be set to exception type or a component-specific low cardinality error code. + +If response status code was sent or received and status indicates an error according to [HTTP span status definition](/docs/http/http-spans.md), +`error.type` SHOULD be set to the status code number (represented as a string), an exception type (if thrown) or a component-specific error code. + +The `error.type` value SHOULD be predictable and SHOULD have low cardinality. +Instrumentations SHOULD document the list of errors they report. + +The cardinality of `error.type` within one instrumentation library SHOULD be low, but +telemetry consumers that aggregate data from multiple instrumentation libraries and applications +should be prepared for `error.type` to have high cardinality at query time, when no +additional filters are applied. + +If the request has completed successfully, instrumentations SHOULD NOT set `error.type`. + +**[2]:** HTTP request method value SHOULD be "known" to the instrumentation. +By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) +and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + +If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + +If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override +the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named +OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods +(this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + +HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. +Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. +Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + +**[3]:** MUST NOT be populated when this is not supported by the HTTP server framework as the route attribute should have low-cardinality and the URI path can NOT substitute it. +SHOULD include the [application root](/docs/http/http-spans.md#http-server-definitions) if there is one. + +**[4]:** The value SHOULD be normalized to lowercase. + +**[5]:** `network.protocol.version` refers to the version of the protocol used and might be different from the protocol client's version. If the HTTP client used has a version of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. + +**[6]:** See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). +> **Warning** +> Since this attribute is based on HTTP headers, opting in to it may allow an attacker +> to trigger cardinality limits, degrading the usefulness of the metric. + +**[7]:** See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). +> **Warning** +> Since this attribute is based on HTTP headers, opting in to it may allow an attacker +> to trigger cardinality limits, degrading the usefulness of the metric. + +`error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `_OTHER` | A fallback error value to be used when the instrumentation does not define a custom value for it. | + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | + + +## HTTP Client + +### Metric: `http.client.request.duration` + +This metric is required. + +When this metric is reported alongside an HTTP client span, the metric value SHOULD be the same as the HTTP client span duration. + +This metric SHOULD be specified with +[`ExplicitBucketBoundaries`](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/metrics/api.md#instrument-advisory-parameters) +of `[ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10 ]`. + + +| Name | Instrument Type | Unit (UCUM) | Description | +| -------- | --------------- | ----------- | -------------- | +| `http.client.request.duration` | Histogram | `s` | Duration of HTTP client requests. | + + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `error.type` | string | Describes a class of error the operation ended with. [1] | `timeout`; `name_resolution_error`; `500` | Conditionally Required: If request has ended with an error. | +| `http.request.method` | string | HTTP request method. [2] | `GET`; `POST`; `HEAD` | Required | +| `http.response.status_code` | int | [HTTP response status code](https://tools.ietf.org/html/rfc7231#section-6). | `200` | Conditionally Required: If and only if one was received/sent. | +| [`network.protocol.name`](../general/attributes.md) | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [3] | `http`; `spdy` | Recommended: if not default (`http`). | +| [`network.protocol.version`](../general/attributes.md) | string | Version of the protocol specified in `network.protocol.name`. [4] | `1.0`; `1.1`; `2`; `3` | Recommended | +| [`server.address`](../general/attributes.md) | string | Host identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [5] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Required | +| [`server.port`](../general/attributes.md) | int | Port identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [6] | `80`; `8080`; `443` | Conditionally Required: [7] | + +**[1]:** If the request fails with an error before response status code was sent or received, +`error.type` SHOULD be set to exception type or a component-specific low cardinality error code. + +If response status code was sent or received and status indicates an error according to [HTTP span status definition](/docs/http/http-spans.md), +`error.type` SHOULD be set to the status code number (represented as a string), an exception type (if thrown) or a component-specific error code. + +The `error.type` value SHOULD be predictable and SHOULD have low cardinality. +Instrumentations SHOULD document the list of errors they report. + +The cardinality of `error.type` within one instrumentation library SHOULD be low, but +telemetry consumers that aggregate data from multiple instrumentation libraries and applications +should be prepared for `error.type` to have high cardinality at query time, when no +additional filters are applied. + +If the request has completed successfully, instrumentations SHOULD NOT set `error.type`. + +**[2]:** HTTP request method value SHOULD be "known" to the instrumentation. +By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) +and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + +If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + +If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override +the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named +OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods +(this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + +HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. +Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. +Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + +**[3]:** The value SHOULD be normalized to lowercase. + +**[4]:** `network.protocol.version` refers to the version of the protocol used and might be different from the protocol client's version. If the HTTP client used has a version of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. + +**[5]:** Determined by using the first of the following that applies + +- Host identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form +- Host identifier of the `Host` header + +If an HTTP client request is explicitly made to an IP address, e.g. `http://x.x.x.x:8080`, then +`server.address` SHOULD be the IP address `x.x.x.x`. A DNS lookup SHOULD NOT be used. + +**[6]:** When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `server.port` MUST match URI port identifier, otherwise it MUST match `Host` header port identifier. + +**[7]:** If not default (`80` for `http` scheme, `443` for `https`). + +`error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `_OTHER` | A fallback error value to be used when the instrumentation does not define a custom value for it. | + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | + + +### Metric: `http.client.request.body.size` + +This metric is optional. + + +| Name | Instrument Type | Unit (UCUM) | Description | +| -------- | --------------- | ----------- | -------------- | +| `http.client.request.body.size` | Histogram | `By` | Size of HTTP client request bodies. [1] | + +**[1]:** The size of the request payload body in bytes. This is the number of bytes transferred excluding headers and is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) header. For requests using transport encoding, this should be the compressed size. + + +### Metric: `http.client.response.body.size` + +This metric is optional. + + +| Name | Instrument Type | Unit (UCUM) | Description | +| -------- | --------------- | ----------- | -------------- | +| `http.client.response.body.size` | Histogram | `By` | Size of HTTP client response bodies. [1] | + +**[1]:** The size of the response payload body in bytes. This is the number of bytes transferred excluding headers and is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) header. For requests using transport encoding, this should be the compressed size. + + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `error.type` | string | Describes a class of error the operation ended with. [1] | `timeout`; `name_resolution_error`; `500` | Conditionally Required: If request has ended with an error. | +| `http.request.method` | string | HTTP request method. [2] | `GET`; `POST`; `HEAD` | Required | +| `http.response.status_code` | int | [HTTP response status code](https://tools.ietf.org/html/rfc7231#section-6). | `200` | Conditionally Required: If and only if one was received/sent. | +| [`network.protocol.name`](../general/attributes.md) | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [3] | `http`; `spdy` | Recommended: if not default (`http`). | +| [`network.protocol.version`](../general/attributes.md) | string | Version of the protocol specified in `network.protocol.name`. [4] | `1.0`; `1.1`; `2`; `3` | Recommended | +| [`server.address`](../general/attributes.md) | string | Host identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [5] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Required | +| [`server.port`](../general/attributes.md) | int | Port identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [6] | `80`; `8080`; `443` | Conditionally Required: [7] | + +**[1]:** If the request fails with an error before response status code was sent or received, +`error.type` SHOULD be set to exception type or a component-specific low cardinality error code. + +If response status code was sent or received and status indicates an error according to [HTTP span status definition](/docs/http/http-spans.md), +`error.type` SHOULD be set to the status code number (represented as a string), an exception type (if thrown) or a component-specific error code. + +The `error.type` value SHOULD be predictable and SHOULD have low cardinality. +Instrumentations SHOULD document the list of errors they report. + +The cardinality of `error.type` within one instrumentation library SHOULD be low, but +telemetry consumers that aggregate data from multiple instrumentation libraries and applications +should be prepared for `error.type` to have high cardinality at query time, when no +additional filters are applied. + +If the request has completed successfully, instrumentations SHOULD NOT set `error.type`. + +**[2]:** HTTP request method value SHOULD be "known" to the instrumentation. +By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) +and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + +If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + +If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override +the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named +OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods +(this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + +HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. +Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. +Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + +**[3]:** The value SHOULD be normalized to lowercase. + +**[4]:** `network.protocol.version` refers to the version of the protocol used and might be different from the protocol client's version. If the HTTP client used has a version of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. + +**[5]:** Determined by using the first of the following that applies + +- Host identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form +- Host identifier of the `Host` header + +If an HTTP client request is explicitly made to an IP address, e.g. `http://x.x.x.x:8080`, then +`server.address` SHOULD be the IP address `x.x.x.x`. A DNS lookup SHOULD NOT be used. + +**[6]:** When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `server.port` MUST match URI port identifier, otherwise it MUST match `Host` header port identifier. + +**[7]:** If not default (`80` for `http` scheme, `443` for `https`). + +`error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `_OTHER` | A fallback error value to be used when the instrumentation does not define a custom value for it. | + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | + + +[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md diff --git a/docs/http/http-spans.md b/docs/http/http-spans.md new file mode 100644 index 0000000..d0b8b94 --- /dev/null +++ b/docs/http/http-spans.md @@ -0,0 +1,563 @@ +# Semantic Conventions for HTTP Spans + +**Status**: [Experimental, Feature-freeze][DocumentStatus] + +This document defines semantic conventions for HTTP client and server Spans. +They can be used for http and https schemes +and various HTTP versions like 1.1, 2 and SPDY. + + + + + +- [Name](#name) +- [Status](#status) +- [Common Attributes](#common-attributes) +- [HTTP client](#http-client) + * [HTTP client span duration](#http-client-span-duration) + * [HTTP request retries and redirects](#http-request-retries-and-redirects) +- [HTTP server](#http-server) + * [HTTP server definitions](#http-server-definitions) + * [HTTP Server semantic conventions](#http-server-semantic-conventions) +- [Examples](#examples) + * [HTTP client-server example](#http-client-server-example) + * [HTTP client retries examples](#http-client-retries-examples) + * [HTTP client authorization retry examples](#http-client-authorization-retry-examples) + * [HTTP client redirects examples](#http-client-redirects-examples) + * [HTTP client call: DNS error](#http-client-call-dns-error) + * [HTTP client call: Internal Server Error](#http-client-call-internal-server-error) + * [HTTP server call: connection dropped before response body was sent](#http-server-call-connection-dropped-before-response-body-was-sent) + + + +## Name + +HTTP spans MUST follow the overall [guidelines for span names](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/trace/api.md#span). + + + +HTTP server span names SHOULD be `{method} {http.route}` if there is a +(low-cardinality) `http.route` available (see below for the exact definition of the [`{method}`](#method-placeholder) placeholder). + +If there is no (low-cardinality) `http.route` available, HTTP server span names +SHOULD be [`{method}`](#method-placeholder). + +HTTP client spans have no `http.route` attribute since client-side instrumentation +is not generally aware of the "route", and therefore HTTP client spans SHOULD be +[`{method}`](#method-placeholder). + + +The `{method}` MUST be `{http.request.method}` if the method represents the original method known to the instrumentation. +In other cases (when `{http.request.method}` is set to `_OTHER`), `{method}` MUST be `HTTP`. + +Instrumentation MUST NOT default to using URI +path as span name, but MAY provide hooks to allow custom logic to override the +default span name. + +## Status + +[Span Status](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/trace/api.md#set-status) MUST be left unset if HTTP status code was in the +1xx, 2xx or 3xx ranges, unless there was another error (e.g., network error receiving +the response body; or 3xx codes with max redirects exceeded), in which case status +MUST be set to `Error`. + +For HTTP status codes in the 4xx range span status MUST be left unset in case of `SpanKind.SERVER` +and MUST be set to `Error` in case of `SpanKind.CLIENT`. + +For HTTP status codes in the 5xx range, as well as any other code the client +failed to interpret, span status MUST be set to `Error`. + +Don't set the span status description if the reason can be inferred from `http.response.status_code`. + +HTTP request may fail if it was cancelled or an error occurred preventing +the client or server from sending/receiving the request/response fully. + +When instrumentation detects such errors it MUST set span status to `Error` +and MUST set the `error.type` attribute. + +## Common Attributes + +The common attributes listed in this section apply to both HTTP clients and servers in addition to +the specific attributes listed in the [HTTP client](#http-client) and [HTTP server](#http-server) +sections below. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `error.type` | string | Describes a class of error the operation ended with. [1] | `timeout`; `name_resolution_error`; `500` | Conditionally Required: If request has ended with an error. | +| `http.request.body.size` | int | The size of the request payload body in bytes. This is the number of bytes transferred excluding headers and is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) header. For requests using transport encoding, this should be the compressed size. | `3495` | Recommended | +| `http.request.header.` | string[] | HTTP request headers, `` being the normalized HTTP Header name (lowercase, with `-` characters replaced by `_`), the value being the header values. [2] | `http.request.header.content_type=["application/json"]`; `http.request.header.x_forwarded_for=["1.2.3.4", "1.2.3.5"]` | Opt-In | +| `http.request.method` | string | HTTP request method. [3] | `GET`; `POST`; `HEAD` | Required | +| `http.request.method_original` | string | Original HTTP method sent by the client in the request line. | `GeT`; `ACL`; `foo` | Conditionally Required: [4] | +| `http.response.body.size` | int | The size of the response payload body in bytes. This is the number of bytes transferred excluding headers and is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) header. For requests using transport encoding, this should be the compressed size. | `3495` | Recommended | +| `http.response.header.` | string[] | HTTP response headers, `` being the normalized HTTP Header name (lowercase, with `-` characters replaced by `_`), the value being the header values. [5] | `http.response.header.content_type=["application/json"]`; `http.response.header.my_custom_header=["abc", "def"]` | Opt-In | +| `http.response.status_code` | int | [HTTP response status code](https://tools.ietf.org/html/rfc7231#section-6). | `200` | Conditionally Required: If and only if one was received/sent. | +| [`network.protocol.name`](../general/attributes.md) | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [6] | `http`; `spdy` | Recommended: if not default (`http`). | +| [`network.protocol.version`](../general/attributes.md) | string | Version of the protocol specified in `network.protocol.name`. [7] | `1.0`; `1.1`; `2`; `3` | Recommended | +| [`network.transport`](../general/attributes.md) | string | [OSI transport layer](https://osi-model.com/transport-layer/) or [inter-process communication method](https://en.wikipedia.org/wiki/Inter-process_communication). [8] | `tcp`; `udp` | Conditionally Required: [9] | +| [`network.type`](../general/attributes.md) | string | [OSI network layer](https://osi-model.com/network-layer/) or non-OSI equivalent. [10] | `ipv4`; `ipv6` | Recommended | +| `user_agent.original` | string | Value of the [HTTP User-Agent](https://www.rfc-editor.org/rfc/rfc9110.html#field.user-agent) header sent by the client. | `CERN-LineMode/2.15 libwww/2.17b3` | Recommended | + +**[1]:** If the request fails with an error before response status code was sent or received, +`error.type` SHOULD be set to exception type or a component-specific low cardinality error code. + +If response status code was sent or received and status indicates an error according to [HTTP span status definition](/docs/http/http-spans.md), +`error.type` SHOULD be set to the status code number (represented as a string), an exception type (if thrown) or a component-specific error code. + +The `error.type` value SHOULD be predictable and SHOULD have low cardinality. +Instrumentations SHOULD document the list of errors they report. + +The cardinality of `error.type` within one instrumentation library SHOULD be low, but +telemetry consumers that aggregate data from multiple instrumentation libraries and applications +should be prepared for `error.type` to have high cardinality at query time, when no +additional filters are applied. + +If the request has completed successfully, instrumentations SHOULD NOT set `error.type`. + +**[2]:** Instrumentations SHOULD require an explicit configuration of which headers are to be captured. Including all request headers can be a security risk - explicit configuration helps avoid leaking sensitive information. +The `User-Agent` header is already captured in the `user_agent.original` attribute. Users MAY explicitly configure instrumentations to capture them even though it is not recommended. +The attribute value MUST consist of either multiple header values as an array of strings or a single-item array containing a possibly comma-concatenated string, depending on the way the HTTP library provides access to headers. + +**[3]:** HTTP request method value SHOULD be "known" to the instrumentation. +By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) +and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + +If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + +If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override +the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named +OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods +(this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + +HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. +Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. +Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + +**[4]:** If and only if it's different than `http.request.method`. + +**[5]:** Instrumentations SHOULD require an explicit configuration of which headers are to be captured. Including all response headers can be a security risk - explicit configuration helps avoid leaking sensitive information. +Users MAY explicitly configure instrumentations to capture them even though it is not recommended. +The attribute value MUST consist of either multiple header values as an array of strings or a single-item array containing a possibly comma-concatenated string, depending on the way the HTTP library provides access to headers. + +**[6]:** The value SHOULD be normalized to lowercase. + +**[7]:** `network.protocol.version` refers to the version of the protocol used and might be different from the protocol client's version. If the HTTP client used has a version of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. + +**[8]:** The value SHOULD be normalized to lowercase. + +Consider always setting the transport when setting a port number, since +a port number is ambiguous without knowing the transport, for example +different processes could be listening on TCP port 12345 and UDP port 12345. + +**[9]:** If not default (`tcp` for `HTTP/1.1` and `HTTP/2`, `udp` for `HTTP/3`). + +**[10]:** The value SHOULD be normalized to lowercase. + +Following attributes MUST be provided **at span creation time** (when provided at all), so they can be considered for sampling decisions: + +* `http.request.method` + +`error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `_OTHER` | A fallback error value to be used when the instrumentation does not define a custom value for it. | + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | + +`network.transport` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `tcp` | TCP | +| `udp` | UDP | +| `pipe` | Named or anonymous pipe. See note below. | +| `unix` | Unix domain socket | + +`network.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `ipv4` | IPv4 | +| `ipv6` | IPv6 | + + +## HTTP client + +This span type represents an outbound HTTP request. There are two ways this can be achieved in an instrumentation: + +1. Instrumentations SHOULD create an HTTP span for each attempt to send an HTTP request over the wire. + In case the request is resent, the resend attempts MUST follow the [HTTP resend spec](#http-request-retries-and-redirects). + In this case, instrumentations SHOULD NOT (also) emit a logical encompassing HTTP client span. + +2. If for some reason it is not possible to emit a span for each send attempt (because e.g. the instrumented library does not expose hooks that would allow this), + instrumentations MAY create an HTTP span for the top-most operation of the HTTP client. + In this case, the `url.full` MUST be the absolute URL that was originally requested, before any HTTP-redirects that may happen when executing the request. + +For an HTTP client span, `SpanKind` MUST be `Client`. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `http.resend_count` | int | The ordinal number of request resending attempt (for any reason, including redirects). [1] | `3` | Recommended: if and only if request was retried. | +| [`network.peer.address`](../general/attributes.md) | string | Peer address of the network connection - IP address or Unix domain socket name. | `10.1.2.80`; `/tmp/my.sock` | Recommended: If different than `server.address`. | +| [`network.peer.port`](../general/attributes.md) | int | Peer port number of the network connection. | `65123` | Recommended: If `network.peer.address` is set. | +| [`server.address`](../general/attributes.md) | string | Host identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [2] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Required | +| [`server.port`](../general/attributes.md) | int | Port identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [3] | `80`; `8080`; `443` | Conditionally Required: [4] | +| `url.full` | string | Absolute URL describing a network resource according to [RFC3986](https://www.rfc-editor.org/rfc/rfc3986) [5] | `https://www.foo.bar/search?q=OpenTelemetry#SemConv`; `//localhost` | Required | + +**[1]:** The resend count SHOULD be updated each time an HTTP request gets resent by the client, regardless of what was the cause of the resending (e.g. redirection, authorization failure, 503 Server Unavailable, network issues, or any other). + +**[2]:** Determined by using the first of the following that applies + +- Host identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form +- Host identifier of the `Host` header + +If an HTTP client request is explicitly made to an IP address, e.g. `http://x.x.x.x:8080`, then +`server.address` SHOULD be the IP address `x.x.x.x`. A DNS lookup SHOULD NOT be used. + +**[3]:** When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `server.port` MUST match URI port identifier, otherwise it MUST match `Host` header port identifier. + +**[4]:** If not default (`80` for `http` scheme, `443` for `https`). + +**[5]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless. +`url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`. +`url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) and SHOULD NOT be validated or modified except for sanitizing purposes. + +Following attributes MUST be provided **at span creation time** (when provided at all), so they can be considered for sampling decisions: + +* [`server.address`](../general/attributes.md) +* [`server.port`](../general/attributes.md) +* `url.full` + + +Note that in some cases host and port identifiers in the `Host` header might be different from the `server.address` and `server.port`, in this case instrumentation MAY populate `Host` header on `http.request.header.host` attribute even if it's not enabled by user. + +### HTTP client span duration + +There are some minimal constraints that SHOULD be honored: + +* HTTP client spans SHOULD start sometime before the first request byte is sent. This may or may not include connection time. +* HTTP client spans SHOULD end sometime after the HTTP response headers are fully read (or when they fail to be read). This may or may not include reading the response body. + +If there is any possibility for application code to not fully read the HTTP response +(and for the HTTP client library to then have to clean up the HTTP response asynchronously), +the HTTP client span SHOULD NOT be ended in this cleanup phase, +and instead SHOULD end at some point after the HTTP response headers are fully read (or fail to be read). +This avoids the span being ended asynchronously later on at a time +which is no longer directly associated with the application code which made the HTTP request. + +Because of the potential for confusion around this, HTTP client library instrumentations SHOULD document their behavior around ending HTTP client spans. + +### HTTP request retries and redirects + +Retries and redirects cause more than one physical HTTP request to be sent. +A request is resent when an HTTP client library sends more than one HTTP request to satisfy the same API call. +This may happen due to following redirects, authorization challenges, 503 Server Unavailable, network issues, or any other. + +Each time an HTTP request is resent, the `http.resend_count` attribute SHOULD be added to each repeated span and set to the ordinal number of the request resend attempt. + +See the examples for more details about: + +* [retrying a server error](#http-client-retries-examples), +* [redirects](#http-client-redirects-examples), +* [authorization](#http-client-authorization-retry-examples). + +## HTTP server + +To understand the attributes defined in this section, it is helpful to read the "Definitions" subsection. + +### HTTP server definitions + +This section gives a short summary of some concepts +in web server configuration and web app deployment +that are relevant to tracing. + +Usually, on a physical host, reachable by one or multiple IP addresses, a single HTTP listener process runs. +If multiple processes are running, they must listen on distinct TCP/UDP ports so that the OS can route incoming TCP/UDP packets to the right one. + +Within a single server process, there can be multiple **virtual hosts**. +The [HTTP host header][] (in combination with a port number) is normally used to determine to which of them to route incoming HTTP requests. + +The host header value that matches some virtual host is called the virtual hosts's **server name**. If there are multiple aliases for the virtual host, one of them (often the first one listed in the configuration) is called the **primary server name**. See for example, the Apache [`ServerName`][ap-sn] or NGINX [`server_name`][nx-sn] directive or the CGI specification on `SERVER_NAME` ([RFC 3875][rfc-servername]). +In practice the HTTP host header is often ignored when just a single virtual host is configured for the IP. + +Within a single virtual host, some servers support the concepts of an **HTTP application** +(for example in Java, the Servlet JSR defines an application as +"a collection of servlets, HTML pages, classes, and other resources that make up a complete application on a Web server" +-- SRV.9 in [JSR 53][]; +in a deployment of a Python application to Apache, the application would be the [PEP 3333][] conformant callable that is configured using the +[`WSGIScriptAlias` directive][modwsgisetup] of `mod_wsgi`). + +An application can be "mounted" under an **application root** +(also known as a *[context root][]*, *[context prefix][]*, or *[document base][]*) +which is a fixed path prefix of the URL that determines to which application a request is routed +(e.g., the server could be configured to route all requests that go to an URL path starting with `/webshop/` +at a particular virtual host +to the `com.example.webshop` web application). + +Some servers allow to bind the same HTTP application to multiple `(virtual host, application root)` pairs. + +> TODO: Find way to trace HTTP application and application root ([opentelemetry/opentelementry-specification#335][]) + +[HTTP host header]: https://tools.ietf.org/html/rfc7230#section-5.4 +[PEP 3333]: https://www.python.org/dev/peps/pep-3333/ +[modwsgisetup]: https://modwsgi.readthedocs.io/en/develop/user-guides/quick-configuration-guide.html +[context root]: https://docs.jboss.org/jbossas/guides/webguide/r2/en/html/ch06.html +[context prefix]: https://marc.info/?l=apache-cvs&m=130928191414740 +[document base]: http://tomcat.apache.org/tomcat-5.5-doc/config/context.html +[rfc-servername]: https://tools.ietf.org/html/rfc3875#section-4.1.14 +[ap-sn]: https://httpd.apache.org/docs/2.4/mod/core.html#servername +[nx-sn]: http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name +[JSR 53]: https://jcp.org/aboutJava/communityprocess/maintenance/jsr053/index2.html +[opentelemetry/opentelementry-specification#335]: https://github.com/open-telemetry/opentelemetry-specification/issues/335 + +### HTTP Server semantic conventions + +This span type represents an inbound HTTP request. + +For an HTTP server span, `SpanKind` MUST be `Server`. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| [`client.address`](../general/attributes.md) | string | Client address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. [1] | `83.164.160.102` | Recommended | +| [`client.port`](../general/attributes.md) | int | The port of the original client behind all proxies, if known (e.g. from [Forwarded](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded) or a similar header). Otherwise, the immediate client peer port. [2] | `65123` | Recommended | +| `http.route` | string | The matched route (path template in the format used by the respective server framework). See note below [3] | `/users/:userID?`; `{controller}/{action}/{id?}` | Conditionally Required: If and only if it's available | +| [`network.local.address`](../general/attributes.md) | string | Local socket address. Useful in case of a multi-IP host. | `10.1.2.80`; `/tmp/my.sock` | Opt-In | +| [`network.local.port`](../general/attributes.md) | int | Local socket port. Useful in case of a multi-port host. | `65123` | Opt-In | +| [`network.peer.address`](../general/attributes.md) | string | Peer address of the network connection - IP address or Unix domain socket name. | `10.1.2.80`; `/tmp/my.sock` | Recommended | +| [`network.peer.port`](../general/attributes.md) | int | Peer port number of the network connection. | `65123` | Recommended | +| [`server.address`](../general/attributes.md) | string | Name of the local HTTP server that received the request. [4] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Recommended | +| [`server.port`](../general/attributes.md) | int | Port of the local HTTP server that received the request. [5] | `80`; `8080`; `443` | Recommended: [6] | +| `url.path` | string | The [URI path](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) component [7] | `/search` | Required | +| `url.query` | string | The [URI query](https://www.rfc-editor.org/rfc/rfc3986#section-3.4) component [8] | `q=OpenTelemetry` | Conditionally Required: If and only if one was received/sent. | +| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `http`; `https` | Required | + +**[1]:** The IP address of the original client behind all proxies, if known (e.g. from [Forwarded](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded), [X-Forwarded-For](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For), or a similar header). Otherwise, the immediate client peer address. + +**[2]:** When observed from the server side, and when communicating through an intermediary, `client.port` SHOULD represent the client port behind any intermediaries (e.g. proxies) if it's available. + +**[3]:** MUST NOT be populated when this is not supported by the HTTP server framework as the route attribute should have low-cardinality and the URI path can NOT substitute it. +SHOULD include the [application root](/docs/http/http-spans.md#http-server-definitions) if there is one. + +**[4]:** Determined by using the first of the following that applies + +- The [primary server name](/docs/http/http-spans.md#http-server-definitions) of the matched virtual host. MUST only + include host identifier. +- Host identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form. +- Host identifier of the `Host` header + +SHOULD NOT be set if only IP address is available and capturing name would require a reverse DNS lookup. + +**[5]:** Determined by using the first of the following that applies + +- Port identifier of the [primary server host](/docs/http/http-spans.md#http-server-definitions) of the matched virtual host. +- Port identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form. +- Port identifier of the `Host` header + +**[6]:** If not default (`80` for `http` scheme, `443` for `https`). + +**[7]:** When missing, the value is assumed to be `/` + +**[8]:** Sensitive content provided in query string SHOULD be scrubbed when instrumentations can identify it. + +Following attributes MUST be provided **at span creation time** (when provided at all), so they can be considered for sampling decisions: + +* [`server.address`](../general/attributes.md) +* [`server.port`](../general/attributes.md) +* `url.path` +* `url.query` +* `url.scheme` + + +`http.route` MUST be provided at span creation time if and only if it's already available. If it becomes available after span starts, instrumentation MUST populate it anytime before span ends. + +Note that in some cases host and port identifiers in the `Host` header might be different from the `server.address` and `server.port`, in this case instrumentation MAY populate `Host` header on `http.request.header.host` attribute even if it's not enabled by user. + +## Examples + +### HTTP client-server example + +As an example, if a browser request for `https://example.com:8080/webshop/articles/4?s=1` is invoked from a host with IP 192.0.2.4, we may have the following Span on the client side: + +Span name: `GET` + +| Attribute name | Value | +| :------------------- | :-------------------------------------------------------| +| `http.request.method`| `"GET"` | +| `network.protocol.version` | `"1.1"` | +| `url.full` | `"https://example.com:8080/webshop/articles/4?s=1"` | +| `server.address` | `example.com` | +| `server.port` | 8080 | +| `server.socket.address` | `"192.0.2.5"` | +| `http.response.status_code` | `200` | + +The corresponding server Span may look like this: + +Span name: `GET /webshop/articles/:article_id`. + +| Attribute name | Value | +| :------------------- | :---------------------------------------------- | +| `http.request.method`| `"GET"` | +| `network.protocol.version` | `"1.1"` | +| `url.path` | `"/webshop/articles/4"` | +| `url.query` | `"?s=1"` | +| `server.address` | `"example.com"` | +| `server.port` | `8080` | +| `url.scheme` | `"https"` | +| `http.route` | `"/webshop/articles/:article_id"` | +| `http.response.status_code` | `200` | +| `client.address` | `"192.0.2.4"` | +| `client.socket.address` | `"192.0.2.5"` (the client goes through a proxy) | +| `user_agent.original` | `"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0"` | + +### HTTP client retries examples + +Example of retries in the presence of a trace started by an inbound request: + +``` +request (SERVER, trace=t1, span=s1) + | + -- GET / - 500 (CLIENT, trace=t1, span=s2) + | | + | --- server (SERVER, trace=t1, span=s3) + | + -- GET / - 500 (CLIENT, trace=t1, span=s4, http.resend_count=1) + | | + | --- server (SERVER, trace=t1, span=s5) + | + -- GET / - 200 (CLIENT, trace=t1, span=s6, http.resend_count=2) + | + --- server (SERVER, trace=t1, span=s7) +``` + +Example of retries with no trace started upfront: + +``` +GET / - 500 (CLIENT, trace=t1, span=s1) + | + --- server (SERVER, trace=t1, span=s2) + +GET / - 500 (CLIENT, trace=t2, span=s1, http.resend_count=1) + | + --- server (SERVER, trace=t2, span=s2) + +GET / - 200 (CLIENT, trace=t3, span=s1, http.resend_count=2) + | + --- server (SERVER, trace=t3, span=s1) +``` + +### HTTP client authorization retry examples + +Example of retries in the presence of a trace started by an inbound request: + +``` +request (SERVER, trace=t1, span=s1) + | + -- GET /hello - 401 (CLIENT, trace=t1, span=s2) + | | + | --- server (SERVER, trace=t1, span=s3) + | + -- GET /hello - 200 (CLIENT, trace=t1, span=s4, http.resend_count=1) + | + --- server (SERVER, trace=t1, span=s5) +``` + +Example of retries with no trace started upfront: + +``` +GET /hello - 401 (CLIENT, trace=t1, span=s1) + | + --- server (SERVER, trace=t1, span=s2) + +GET /hello - 200 (CLIENT, trace=t2, span=s1, http.resend_count=1) + | + --- server (SERVER, trace=t2, span=s2) +``` + +### HTTP client redirects examples + +Example of redirects in the presence of a trace started by an inbound request: + +``` +request (SERVER, trace=t1, span=s1) + | + -- GET / - 302 (CLIENT, trace=t1, span=s2) + | | + | --- server (SERVER, trace=t1, span=s3) + | + -- GET /hello - 200 (CLIENT, trace=t1, span=s4, http.resend_count=1) + | + --- server (SERVER, trace=t1, span=s5) +``` + +Example of redirects with no trace started upfront: + +``` +GET / - 302 (CLIENT, trace=t1, span=s1) + | + --- server (SERVER, trace=t1, span=s2) + +GET /hello - 200 (CLIENT, trace=t2, span=s1, http.resend_count=1) + | + --- server (SERVER, trace=t2, span=s2) +``` + +### HTTP client call: DNS error + +As an example, if a user requested `https://does-not-exist-123.com`, we may have the following span on the client side: + +| Attribute name | Value | +| :------------------- | :-------------------------------------------------------| +| `http.request.method`| `"GET"` | +| `network.protocol.version` | `"1.1"` | +| `url.full` | `"https://does-not-exist-123.com"` | +| `server.address` | `"does-not-exist-123.com"` | +| `error.type` | `"java.net.UnknownHostException"` | + +### HTTP client call: Internal Server Error + +As an example, if a user requested `https://example.com` and server returned 500, we may have the following span on the client side: + +| Attribute name | Value | +| :------------------- | :-------------------------------------------------------| +| `http.request.method`| `"GET"` | +| `network.protocol.version` | `"1.1"` | +| `url.full` | `"https://example.com"` | +| `server.address` | `"example.com"` | +| `http.response.status_code` | `500` | +| `error.type` | `"500"` | + +### HTTP server call: connection dropped before response body was sent + +As an example, if a user sent a `POST` request with a body to `https://example.com:8080/uploads/4`, we may see the following span on a server side: + +Span name: `POST /uploads/:document_id`. + +| Attribute name | Value | +| :------------------- | :---------------------------------------------- | +| `http.request.method`| `"GET"` | +| `url.path` | `"/uploads/4"` | +| `url.scheme` | `"https"` | +| `http.route` | `"/uploads/:document_id"` | +| `http.response.status_code` | `201` | +| `error.type` | `WebSocketDisconnect` | + +[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md diff --git a/model/client.yaml b/model/client.yaml new file mode 100644 index 0000000..71d285f --- /dev/null +++ b/model/client.yaml @@ -0,0 +1,26 @@ +groups: + - id: client + prefix: client + type: attribute_group + brief: > + These attributes may be used to describe the client in a connection-based network interaction + where there is one side that initiates the connection (the client is the side that initiates the connection). + This covers all TCP network interactions since TCP is connection-based and one side initiates the + connection (an exception is made for peer-to-peer communication over TCP where the "user-facing" surface of the + protocol / API does not expose a clear notion of client and server). + This also covers UDP network interactions where one side initiates the interaction, e.g. QUIC (HTTP/3) and DNS. + attributes: + - id: address + type: string + brief: Client address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. + note: > + When observed from the server side, and when communicating through an intermediary, `client.address` SHOULD represent + the client address behind any intermediaries (e.g. proxies) if it's available. + examples: ['client.example.com', '10.1.2.80', '/tmp/my.sock'] + - id: port + type: int + brief: Client port number. + examples: [65123] + note: > + When observed from the server side, and when communicating through an intermediary, `client.port` SHOULD represent + the client port behind any intermediaries (e.g. proxies) if it's available. diff --git a/model/destination.yaml b/model/destination.yaml new file mode 100644 index 0000000..bad57d5 --- /dev/null +++ b/model/destination.yaml @@ -0,0 +1,22 @@ +groups: + - id: destination + prefix: destination + type: attribute_group + brief: These attributes may be used to describe the receiver of a network exchange/packet. These should be used + when there is no client/server relationship between the two sides, or when that relationship is unknown. + This covers low-level network interactions (e.g. packet tracing) where you don't know if + there was a connection or which side initiated it. + This also covers unidirectional UDP flows and peer-to-peer communication where the + "user-facing" surface of the protocol / API does not expose a clear notion of client and server. + attributes: + - id: address + type: string + brief: Destination address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. + note: > + When observed from the source side, and when communicating through an intermediary, `destination.address` SHOULD represent + the destination address behind any intermediaries (e.g. proxies) if it's available. + examples: ['destination.example.com', '10.1.2.80', '/tmp/my.sock'] + - id: port + type: int + brief: 'Destination port number' + examples: [3389, 2888] diff --git a/model/error.yaml b/model/error.yaml new file mode 100644 index 0000000..cdfc587 --- /dev/null +++ b/model/error.yaml @@ -0,0 +1,31 @@ +groups: + - id: error + type: attribute_group + prefix: error + brief: > + This document defines the shared attributes used to + report an error. + attributes: + - id: type + brief: 'Describes a class of error the operation ended with.' + type: + allow_custom_values: true + members: + - id: other + value: "_OTHER" + brief: 'A fallback error value to be used when the instrumentation does not define a custom value for it.' + examples: ['timeout', 'java.net.UnknownHostException', 'server_certificate_invalid', '500'] + note: | + The `error.type` SHOULD be predictable and SHOULD have low cardinality. + Instrumentations SHOULD document the list of errors they report. + + The cardinality of `error.type` within one instrumentation library SHOULD be low, but + telemetry consumers that aggregate data from multiple instrumentation libraries and applications + should be prepared for `error.type` to have high cardinality at query time, when no + additional filters are applied. + + If the operation has completed successfully, instrumentations SHOULD NOT set `error.type`. + + If a specific domain defines its own set of error codes (such as HTTP or gRPC status codes), + it's RECOMMENDED to use a domain-specific attribute and also set `error.type` to capture + all errors, regardless of whether they are defined within the domain-specific set or not. diff --git a/model/general.yaml b/model/general.yaml new file mode 100644 index 0000000..be45d72 --- /dev/null +++ b/model/general.yaml @@ -0,0 +1,54 @@ +groups: + - id: general.client + type: attribute_group + brief: > + General client attributes. + attributes: + - ref: client.address + - ref: client.port + - id: general.server + type: attribute_group + brief: > + General server attributes. + attributes: + - ref: server.address + - ref: server.port + - id: general.source + type: attribute_group + brief: > + General source attributes. + attributes: + - ref: source.address + - ref: source.port + - id: general.destination + type: attribute_group + brief: > + General destination attributes. + attributes: + - ref: destination.address + - ref: destination.port + - id: peer + prefix: peer + type: span + brief: "Operations that access some remote service." + attributes: + - id: service + type: string + stability: experimental + brief: > + The [`service.name`](/docs/resource/README.md#service) + of the remote service. SHOULD be equal to the actual `service.name` + resource attribute of the remote service if any. + examples: "AuthTokenCache" + - id: code + type: span + brief: > + These attributes allow to report this unit of code and therefore to provide more context about the span. + attributes: + - ref: code.function + - ref: code.namespace + - ref: code.filepath + - ref: code.lineno + - ref: code.column + - ref: code.stacktrace + requirement_level: opt_in diff --git a/model/http-common.yaml b/model/http-common.yaml new file mode 100644 index 0000000..54e53fc --- /dev/null +++ b/model/http-common.yaml @@ -0,0 +1,101 @@ +groups: + - id: attributes.http.common + type: attribute_group + brief: "Describes HTTP attributes." + attributes: + - ref: http.request.method + requirement_level: required + - ref: http.response.status_code + requirement_level: + conditionally_required: If and only if one was received/sent. + - ref: error.type + requirement_level: + conditionally_required: If request has ended with an error. + examples: ['timeout', 'name_resolution_error', '500'] + note: | + If the request fails with an error before response status code was sent or received, + `error.type` SHOULD be set to exception type or a component-specific low cardinality error code. + + If response status code was sent or received and status indicates an error according to [HTTP span status definition](/docs/http/http-spans.md), + `error.type` SHOULD be set to the status code number (represented as a string), an exception type (if thrown) or a component-specific error code. + + The `error.type` value SHOULD be predictable and SHOULD have low cardinality. + Instrumentations SHOULD document the list of errors they report. + + The cardinality of `error.type` within one instrumentation library SHOULD be low, but + telemetry consumers that aggregate data from multiple instrumentation libraries and applications + should be prepared for `error.type` to have high cardinality at query time, when no + additional filters are applied. + + If the request has completed successfully, instrumentations SHOULD NOT set `error.type`. + - ref: network.protocol.name + examples: ['http', 'spdy'] + requirement_level: + recommended: if not default (`http`). + - ref: network.protocol.version + examples: ['1.0', '1.1', '2', '3'] + + - id: attributes.http.client + type: attribute_group + brief: 'HTTP Client attributes' + extends: attributes.http.common + attributes: + - ref: server.address + requirement_level: required + brief: > + Host identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. + note: | + Determined by using the first of the following that applies + + - Host identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form + - Host identifier of the `Host` header + + If an HTTP client request is explicitly made to an IP address, e.g. `http://x.x.x.x:8080`, then + `server.address` SHOULD be the IP address `x.x.x.x`. A DNS lookup SHOULD NOT be used. + - ref: server.port + requirement_level: + conditionally_required: If not default (`80` for `http` scheme, `443` for `https`). + brief: > + Port identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. + note: > + When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `server.port` MUST match + URI port identifier, otherwise it MUST match `Host` header port identifier. + + - id: attributes.http.server + type: attribute_group + brief: 'HTTP Server attributes' + extends: attributes.http.common + attributes: + - ref: http.route + requirement_level: + conditionally_required: If and only if it's available + - ref: server.address + brief: > + Name of the local HTTP server that received the request. + note: | + Determined by using the first of the following that applies + + - The [primary server name](/docs/http/http-spans.md#http-server-definitions) of the matched virtual host. MUST only + include host identifier. + - Host identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form. + - Host identifier of the `Host` header + + SHOULD NOT be set if only IP address is available and capturing name would require a reverse DNS lookup. + + - ref: server.port + brief: > + Port of the local HTTP server that received the request. + note: | + Determined by using the first of the following that applies + + - Port identifier of the [primary server host](/docs/http/http-spans.md#http-server-definitions) of the matched virtual host. + - Port identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) + if it's sent in absolute-form. + - Port identifier of the `Host` header + requirement_level: + recommended: If not default (`80` for `http` scheme, `443` for `https`). + - ref: url.scheme + requirement_level: required + examples: ["http", "https"] diff --git a/model/metrics/contrast-actions.yaml b/model/metrics/contrast-actions.yaml new file mode 100644 index 0000000..fb9b922 --- /dev/null +++ b/model/metrics/contrast-actions.yaml @@ -0,0 +1,67 @@ +groups: + - id: attributes.contrast.actions + type: attribute_group + brief: "Actions that we track in Contrast" + attributes: + - id: action + brief: > + The type of action that was observed. + requirement_level: required + examples: + - file-open-create + - authn-request + type: + allow_custom_values: false + members: + - id: storage-query + value: "storage-query" + brief: "Functions that execute queries" + - id: file-open-create + value: "file-open-create" + brief: "file open or create action" + - id: url-forward + value: "url-forward" + brief: "Any function designed to forward a request to another URL" + - id: url-redirect + value: "url-redirect" + brief: "Function that result in an http 302 redirect code sent to the client" + - id: host-cmd-exec + value: "host-cmd-exec" + brief: "system shell command execution" + - id: ldap-query + value: "ldap-query" + brief: "Functions that result in and ldap query operation" + - id: smtp-exec + value: "smtp-exec" + brief: "Functions that result in an SMTP command execution" + - id: outbound-service-call + value: "outbound-service-call" + brief: "Functions that result in external calls to other services" + - id: authn-request + value: "authn-request" + brief: "Functions that perform authentication actions" + - id: authz-request + value: "authz-request" + brief: "Functions that perform authorization actions" + - id: el-execution + value: "el-execution" + brief: "Spring expression language execution" + - id: ognl-execution + value: "ognl-execution" + brief: "Object-Graph Navigation Language expression execution." + + - id: attributes.contrast.action.metric + extends: attributes.contrast.actions + type: attribute_group + brief: "Describes attributes for Contrast Action metrics" + attributes: + - ref: http.request.method + - ref: http.route + + - id: metric.http.server.action.total + type: metric + extends: attributes.contrast.action.metric + metric_name: http.server.action.total + unit: "{action}" + brief: "A counter of actions for contrast" + instrument: counter diff --git a/model/metrics/http.yaml b/model/metrics/http.yaml new file mode 100644 index 0000000..d22e783 --- /dev/null +++ b/model/metrics/http.yaml @@ -0,0 +1,126 @@ +groups: + - id: metric_attributes.http.server + type: attribute_group + brief: 'HTTP server attributes' + extends: attributes.http.server + attributes: + - ref: server.address + requirement_level: opt_in + note: | + See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). + > **Warning** + > Since this attribute is based on HTTP headers, opting in to it may allow an attacker + > to trigger cardinality limits, degrading the usefulness of the metric. + - ref: server.port + requirement_level: opt_in + note: | + See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). + > **Warning** + > Since this attribute is based on HTTP headers, opting in to it may allow an attacker + > to trigger cardinality limits, degrading the usefulness of the metric. + - id: metric_attributes.http.client + type: attribute_group + brief: 'HTTP client attributes' + extends: attributes.http.client + + - id: metric.http.server.request.duration + type: metric + metric_name: http.server.request.duration + brief: "Duration of HTTP server requests." + instrument: histogram + unit: "s" + stability: stable + extends: metric_attributes.http.server + + - id: metric.http.server.active_requests + type: metric + metric_name: http.server.active_requests + stability: experimental + brief: "Number of active HTTP server requests." + instrument: updowncounter + unit: "{request}" + attributes: + - ref: http.request.method + requirement_level: required + - ref: url.scheme + requirement_level: required + examples: ["http", "https"] + - ref: server.address + requirement_level: opt_in + brief: > + Name of the local HTTP server that received the request. + note: | + See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). + > **Warning** + > Since this attribute is based on HTTP headers, opting in to it may allow an attacker + > to trigger cardinality limits, degrading the usefulness of the metric. + - ref: server.port + requirement_level: opt_in + brief: > + Port of the local HTTP server that received the request. + note: | + See [Setting `server.address` and `server.port` attributes](/docs/http/http-spans.md#setting-serveraddress-and-serverport-attributes). + > **Warning** + > Since this attribute is based on HTTP headers, opting in to it may allow an attacker + > to trigger cardinality limits, degrading the usefulness of the metric. + + - id: metric.http.server.request.body.size + type: metric + metric_name: http.server.request.body.size + stability: experimental + brief: "Size of HTTP server request bodies." + instrument: histogram + unit: "By" + note: > + The size of the request payload body in bytes. This is the number of bytes transferred excluding headers and + is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) + header. For requests using transport encoding, this should be the compressed size. + extends: metric_attributes.http.server + + - id: metric.http.server.response.body.size + type: metric + metric_name: http.server.response.body.size + stability: experimental + brief: "Size of HTTP server response bodies." + instrument: histogram + unit: "By" + note: > + The size of the response payload body in bytes. This is the number of bytes transferred excluding headers and + is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) + header. For requests using transport encoding, this should be the compressed size. + extends: metric_attributes.http.server + + - id: metric.http.client.request.duration + type: metric + metric_name: http.client.request.duration + brief: "Duration of HTTP client requests." + instrument: histogram + unit: "s" + stability: stable + extends: metric_attributes.http.client + + - id: metric.http.client.request.body.size + type: metric + metric_name: http.client.request.body.size + stability: experimental + brief: "Size of HTTP client request bodies." + instrument: histogram + unit: "By" + note: > + The size of the request payload body in bytes. This is the number of bytes transferred excluding headers and + is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) + header. For requests using transport encoding, this should be the compressed size. + extends: metric_attributes.http.client + + - id: metric.http.client.response.body.size + type: metric + metric_name: http.client.response.body.size + stability: experimental + brief: "Size of HTTP client response bodies." + instrument: histogram + unit: "By" + note: > + The size of the response payload body in bytes. This is the number of bytes transferred excluding headers and + is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) + header. For requests using transport encoding, this should be the compressed size. + extends: metric_attributes.http.client diff --git a/model/network.yaml b/model/network.yaml new file mode 100644 index 0000000..819daf3 --- /dev/null +++ b/model/network.yaml @@ -0,0 +1,75 @@ +groups: + - id: network-core + prefix: network + type: attribute_group + brief: > + These attributes may be used for any network related operation. + attributes: + - id: transport + type: + allow_custom_values: true + members: + - id: tcp + value: 'tcp' + brief: "TCP" + - id: udp + value: 'udp' + brief: "UDP" + - id: pipe + value: "pipe" + brief: 'Named or anonymous pipe. See note below.' + - id: unix + value: 'unix' + brief: "Unix domain socket" + brief: > + [OSI transport layer](https://osi-model.com/transport-layer/) or + [inter-process communication method](https://en.wikipedia.org/wiki/Inter-process_communication). + note: | + The value SHOULD be normalized to lowercase. + + Consider always setting the transport when setting a port number, since + a port number is ambiguous without knowing the transport, for example + different processes could be listening on TCP port 12345 and UDP port 12345. + examples: ['tcp', 'udp'] + - id: type + type: + allow_custom_values: true + members: + - id: ipv4 + value: 'ipv4' + brief: "IPv4" + - id: ipv6 + value: 'ipv6' + brief: "IPv6" + brief: '[OSI network layer](https://osi-model.com/network-layer/) or non-OSI equivalent.' + note: The value SHOULD be normalized to lowercase. + examples: ['ipv4', 'ipv6'] + - id: protocol.name + type: string + brief: '[OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent.' + note: The value SHOULD be normalized to lowercase. + examples: ['amqp', 'http', 'mqtt'] + - id: protocol.version + type: string + brief: Version of the protocol specified in `network.protocol.name`. + examples: '3.1.1' + note: > + `network.protocol.version` refers to the version of the protocol used and might be + different from the protocol client's version. If the HTTP client used has a version + of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. + - id: peer.address + type: string + brief: Peer address of the network connection - IP address or Unix domain socket name. + examples: ['10.1.2.80', '/tmp/my.sock'] + - id: peer.port + type: int + brief: Peer port number of the network connection. + examples: [65123] + - id: local.address + type: string + brief: Local address of the network connection - IP address or Unix domain socket name. + examples: ['10.1.2.80', '/tmp/my.sock'] + - id: local.port + type: int + brief: Local port number of the network connection. + examples: [65123] diff --git a/model/registry/code.yaml b/model/registry/code.yaml new file mode 100644 index 0000000..5848b4f --- /dev/null +++ b/model/registry/code.yaml @@ -0,0 +1,47 @@ +groups: + - id: registry.code + prefix: code + type: span + brief: > + These attributes allow to report this unit of code and therefore to provide more context about the span. + attributes: + - id: function + type: string + stability: experimental + brief: > + The method or function name, or equivalent (usually rightmost part of the code unit's name). + examples: serveRequest + - id: namespace + type: string + stability: experimental + brief: > + The "namespace" within which `code.function` is defined. Usually the qualified class or module name, + such that `code.namespace` + some separator + `code.function` form a unique identifier for the code unit. + examples: com.example.MyHttpService + - id: filepath + type: string + stability: experimental + brief: > + The source code file name that identifies the code unit as uniquely as possible (preferably an absolute file path). + examples: /usr/local/MyApplication/content_root/app/index.php + - id: lineno + type: int + stability: experimental + brief: > + The line number in `code.filepath` best representing the operation. It SHOULD point within the code unit named in `code.function`. + examples: 42 + - id: column + type: int + stability: experimental + brief: > + The column number in `code.filepath` best representing the operation. It SHOULD point within the code unit named in `code.function`. + examples: 16 + - id: stacktrace + type: string + stability: experimental + brief: > + A stacktrace as a string in the natural representation for the language runtime. + The representation is to be determined and documented by each language SIG. + examples: 'at com.example.GenerateTrace.methodB(GenerateTrace.java:13)\n + at com.example.GenerateTrace.methodA(GenerateTrace.java:9)\n + at com.example.GenerateTrace.main(GenerateTrace.java:5)' diff --git a/model/registry/http.yaml b/model/registry/http.yaml new file mode 100644 index 0000000..1d6f019 --- /dev/null +++ b/model/registry/http.yaml @@ -0,0 +1,126 @@ +groups: + - id: registry.http + prefix: http + type: attribute_group + brief: 'This document defines semantic convention attributes in the HTTP namespace.' + attributes: + - id: request.body.size + type: int + brief: > + The size of the request payload body in bytes. This is the number of bytes transferred excluding headers and + is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) + header. For requests using transport encoding, this should be the compressed size. + examples: 3495 + - id: request.header + type: template[string[]] + brief: > + HTTP request headers, `` being the normalized HTTP Header name (lowercase, with `-` characters replaced by `_`), the value being the header values. + note: > + Instrumentations SHOULD require an explicit configuration of which headers are to be captured. + Including all request headers can be a security risk - explicit configuration helps avoid leaking sensitive information. + + The `User-Agent` header is already captured in the `user_agent.original` attribute. + Users MAY explicitly configure instrumentations to capture them even though it is not recommended. + + The attribute value MUST consist of either multiple header values as an array of strings + or a single-item array containing a possibly comma-concatenated string, depending on the way + the HTTP library provides access to headers. + examples: ['http.request.header.content_type=["application/json"]', 'http.request.header.x_forwarded_for=["1.2.3.4", "1.2.3.5"]'] + - id: request.method + type: + allow_custom_values: true + members: + - id: connect + value: "CONNECT" + brief: 'CONNECT method.' + - id: delete + value: "DELETE" + brief: 'DELETE method.' + - id: get + value: "GET" + brief: 'GET method.' + - id: head + value: "HEAD" + brief: 'HEAD method.' + - id: options + value: "OPTIONS" + brief: 'OPTIONS method.' + - id: patch + value: "PATCH" + brief: 'PATCH method.' + - id: post + value: "POST" + brief: 'POST method.' + - id: put + value: "PUT" + brief: 'PUT method.' + - id: trace + value: "TRACE" + brief: 'TRACE method.' + - id: other + value: "_OTHER" + brief: 'Any HTTP method that the instrumentation has no prior knowledge of.' + brief: 'HTTP request method.' + examples: ["GET", "POST", "HEAD"] + note: | + HTTP request method value SHOULD be "known" to the instrumentation. + By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods) + and the PATCH method defined in [RFC5789](https://www.rfc-editor.org/rfc/rfc5789.html). + + If the HTTP request method is not known to instrumentation, it MUST set the `http.request.method` attribute to `_OTHER`. + + If the HTTP instrumentation could end up converting valid HTTP request methods to `_OTHER`, then it MUST provide a way to override + the list of known HTTP methods. If this override is done via environment variable, then the environment variable MUST be named + OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS and support a comma-separated list of case-sensitive known HTTP methods + (this list MUST be a full override of the default known method, it is not a list of known methods in addition to the defaults). + + HTTP method names are case-sensitive and `http.request.method` attribute value MUST match a known HTTP method name exactly. + Instrumentations for specific web frameworks that consider HTTP methods to be case insensitive, SHOULD populate a canonical equivalent. + Tracing instrumentations that do so, MUST also set `http.request.method_original` to the original value. + - id: request.method_original + type: string + brief: Original HTTP method sent by the client in the request line. + examples: ["GeT", "ACL", "foo"] + - id: resend_count + type: int + brief: > + The ordinal number of request resending attempt (for any reason, including redirects). + note: > + The resend count SHOULD be updated each time an HTTP request gets resent by the client, regardless of what + was the cause of the resending (e.g. redirection, authorization failure, 503 Server Unavailable, network issues, + or any other). + examples: 3 + - id: response.body.size + type: int + brief: > + The size of the response payload body in bytes. This is the number of bytes transferred excluding headers and + is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length) + header. For requests using transport encoding, this should be the compressed size. + examples: 3495 + - id: response.header + type: template[string[]] + brief: > + HTTP response headers, `` being the normalized HTTP Header name (lowercase, with `-` characters replaced by `_`), the value being the header values. + note: > + Instrumentations SHOULD require an explicit configuration of which headers are to be captured. + Including all response headers can be a security risk - explicit configuration helps avoid leaking sensitive information. + + Users MAY explicitly configure instrumentations to capture them even though it is not recommended. + + The attribute value MUST consist of either multiple header values as an array of strings + or a single-item array containing a possibly comma-concatenated string, depending on the way + the HTTP library provides access to headers. + examples: ['http.response.header.content_type=["application/json"]', 'http.response.header.my_custom_header=["abc", "def"]'] + - id: response.status_code + type: int + brief: '[HTTP response status code](https://tools.ietf.org/html/rfc7231#section-6).' + examples: [200] + - id: route + type: string + brief: > + The matched route (path template in the format used by the respective server framework). See note below + examples: ['/users/:userID?', '{controller}/{action}/{id?}'] + note: > + MUST NOT be populated when this is not supported by the HTTP server framework as the route attribute should have low-cardinality and the URI path can NOT substitute it. + + SHOULD include the [application root](/docs/http/http-spans.md#http-server-definitions) if there is one. diff --git a/model/server.yaml b/model/server.yaml new file mode 100644 index 0000000..8f10470 --- /dev/null +++ b/model/server.yaml @@ -0,0 +1,26 @@ +groups: + - id: server + prefix: server + type: attribute_group + brief: > + These attributes may be used to describe the server in a connection-based network interaction + where there is one side that initiates the connection (the client is the side that initiates the connection). + This covers all TCP network interactions since TCP is connection-based and one side initiates the + connection (an exception is made for peer-to-peer communication over TCP where the "user-facing" surface of the + protocol / API does not expose a clear notion of client and server). + This also covers UDP network interactions where one side initiates the interaction, e.g. QUIC (HTTP/3) and DNS. + attributes: + - id: address + type: string + brief: Server address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. + note: | + When observed from the client side, and when communicating through an intermediary, `server.address` SHOULD represent + the server address behind any intermediaries (e.g. proxies) if it's available. + examples: ['example.com', '10.1.2.80', '/tmp/my.sock'] + - id: port + type: int + brief: Server port number. + note: > + When observed from the client side, and when communicating through an intermediary, `server.port` SHOULD represent + the server port behind any intermediaries (e.g. proxies) if it's available. + examples: [80, 8080, 443] diff --git a/model/source.yaml b/model/source.yaml new file mode 100644 index 0000000..9c0aa30 --- /dev/null +++ b/model/source.yaml @@ -0,0 +1,22 @@ +groups: + - id: source + prefix: source + type: attribute_group + brief: These attributes may be used to describe the sender of a network exchange/packet. These should be used + when there is no client/server relationship between the two sides, or when that relationship is unknown. + This covers low-level network interactions (e.g. packet tracing) where you don't know if + there was a connection or which side initiated it. + This also covers unidirectional UDP flows and peer-to-peer communication where the + "user-facing" surface of the protocol / API does not expose a clear notion of client and server. + attributes: + - id: address + type: string + brief: Source address - domain name if available without reverse DNS lookup, otherwise IP address or Unix domain socket name. + note: > + When observed from the destination side, and when communicating through an intermediary, `source.address` SHOULD represent + the source address behind any intermediaries (e.g. proxies) if it's available. + examples: ['source.example.com', '10.1.2.80', '/tmp/my.sock'] + - id: port + type: int + brief: 'Source port number' + examples: [3389, 2888] diff --git a/model/contrast-actions.yml b/model/trace/contrast-actions.yml similarity index 67% rename from model/contrast-actions.yml rename to model/trace/contrast-actions.yml index a9f362e..5c1c27e 100644 --- a/model/contrast-actions.yml +++ b/model/trace/contrast-actions.yml @@ -1,85 +1,4 @@ groups: - - id: attributes.contrast.actions - type: attribute_group - brief: "Actions that we track in Contrast" - attributes: - - id: action - brief: > - The type of action that was observed. - requirement_level: required - examples: - - file-open-create - - authn-request - type: - allow_custom_values: false - members: - - id: storage-query - value: "storage-query" - brief: "Functions that execute queries" - - id: file-open-create - value: "file-open-create" - brief: "file open or create action" - - id: url-forward - value: "url-forward" - brief: "Any function designed to forward a request to another URL" - - id: url-redirect - value: "url-redirect" - brief: "Function that result in an http 302 redirect code sent to the client" - - id: host-cmd-exec - value: "host-cmd-exec" - brief: "system shell command execution" - - id: ldap-query - value: "ldap-query" - brief: "Functions that result in and ldap query operation" - - id: smtp-exec - value: "smtp-exec" - brief: "Functions that result in an SMTP command execution" - - id: outbound-service-call - value: "outbound-service-call" - brief: "Functions that result in external calls to other services" - - id: authn-request - value: "authn-request" - brief: "Functions that perform authentication actions" - - id: authz-request - value: "authz-request" - brief: "Functions that perform authorization actions" - - id: el-execution - value: "el-execution" - brief: "Spring expression language execution" - - id: ognl-execution - value: "ognl-execution" - brief: "Object-Graph Navigation Language expression execution." - - - id: attributes.contrast.action.metric - type: attribute_group - brief: "Describes attributes for Contrast Action metrics" - attributes: - - id: http.method - type: string - requirement_level: required - brief: > - http method used when the action was encountered. - examples: - - GET - - POST - - id: http.route - type: string - requirement_level: required - brief: > - http route used when the action was encountered. - examples: "/foo/bar" - - - id: metric.http.server.action.total - type: metric - metric_name: http.server.action.total - unit: "{action}" - brief: "A counter of actions for contrast" - instrument: counter - attributes: - - ref: action - - ref: http.method - - ref: http.route - - id: attributes.contrast.action.span.authn prefix: contrast.authentication type: span @@ -209,3 +128,79 @@ groups: - ref: contrast.authorization.mac.labels requirement_level: recommended: If mechanism is 'mac' + + - id: contrast.action.span.file-open-create + prefix: file.open + type: span + brief: > + Describes attributes for Contrast Action span of type file-open-create. + attributes: + - id: path + brief: > + The absolute path that was accessed. + requirement_level: required + examples: [/etc/myconfig, /foo/bar, /some/tmp] + type: string + - id: flags + brief: > + The flags used when the file was opened or created. + requirement_level: recommended + examples: [o_rdonly, o_rdwr] + type: + allow_custom_values: false + members: + - id: o_rdonly + value: o_rdonly + brief: Read only access + - id: o_wronly + value: o_wronly + brief: Write only access + - id: o_rdwr + value: o_rdwr + brief: Read/write access + + - id: contrast.action.span.host-cmd-exec + type: span + brief: > + Describes attributes for Contrast Action span of type host-cmd-exec. + attributes: + - id: cmd + brief: > + String of executed command with its arguments. + requirement_level: required + examples: [ls /foo, bash -c somebin, chmod 755 foobar] + type: string + + - id: contrast.action.span.outbound-service-call + type: span + extends: trace.http.client + brief: > + Describes attributes for Contrast Action span of type outbound-service-call. + These actions are always using the http protocol. + + - id: attributes.contrast.code-exec + type: attribute_group + brief: Attributes that refer to code execution operations + attributes: + - id: code.contents + requirement_level: required + type: string + brief: The code representing the expression being executed. + examples: [ + "#{'String1 ' + 'string2'}", + "#{20 - 1}", + "'Just a string value'.substring(5)" + ] + - id: contrast.action.span.el-execution + type: span + extends: attributes.contrast.code-exec + brief: > + Describes attributes for Contrast Action span of type el-execution. The Spring Expression Language (SpEL for short) is a + powerful expression language that supports querying and manipulating an object graph at runtime. + - id: contrast.action.span.ognl-execution + extends: attributes.contrast.code-exec + type: span + brief: > + Describes attributes for Contrast Action span of type ognl-execution. OGNL stands for Object-Graph Navigation Language. + It is an expression language for getting and setting properties of Java objects, plus other extras such as list projection + and selection and lambda expressions. You use the same expression for both getting and setting the value of a property. diff --git a/model/trace/http.yaml b/model/trace/http.yaml new file mode 100644 index 0000000..adb3534 --- /dev/null +++ b/model/trace/http.yaml @@ -0,0 +1,90 @@ +groups: + - id: trace.http.common + extends: attributes.http.common + type: attribute_group + brief: 'This document defines semantic conventions for HTTP client and server Spans.' + note: > + These conventions can be used for http and https schemes + and various HTTP versions like 1.1, 2 and SPDY. + attributes: + - ref: http.request.method_original + requirement_level: + conditionally_required: If and only if it's different than `http.request.method`. + - ref: http.request.body.size + - ref: http.request.header + requirement_level: opt_in + - ref: http.response.body.size + - ref: http.response.header + requirement_level: opt_in + - ref: http.request.method + sampling_relevant: true + requirement_level: required + - ref: network.transport + requirement_level: + conditionally_required: If not default (`tcp` for `HTTP/1.1` and `HTTP/2`, `udp` for `HTTP/3`). + - ref: network.type + - ref: user_agent.original + + - id: trace.http.client + type: span + extends: attributes.http.client + span_kind: client + brief: 'Semantic Convention for HTTP Client' + attributes: + - ref: http.resend_count + requirement_level: + recommended: if and only if request was retried. + - ref: server.address + sampling_relevant: true + - ref: server.port + sampling_relevant: true + - ref: network.peer.address + requirement_level: + recommended: If different than `server.address`. + - ref: network.peer.port + requirement_level: + recommended: If `network.peer.address` is set. + - ref: url.full + sampling_relevant: true + requirement_level: required + + - id: trace.http.server + type: span + extends: attributes.http.server + span_kind: server + brief: 'Semantic Convention for HTTP Server' + attributes: + - ref: http.route + - ref: server.address + sampling_relevant: true + - ref: server.port + sampling_relevant: true + - ref: network.local.address + requirement_level: opt_in + brief: Local socket address. Useful in case of a multi-IP host. + - ref: network.local.port + requirement_level: opt_in + brief: Local socket port. Useful in case of a multi-port host. + - ref: client.address + note: > + The IP address of the original client behind all proxies, if + known (e.g. from [Forwarded](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded), + [X-Forwarded-For](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For), or a similar header). + Otherwise, the immediate client peer address. + examples: ['83.164.160.102'] + - ref: client.port + brief: > + The port of the original client behind all proxies, if + known (e.g. from [Forwarded](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded) or a similar header). + Otherwise, the immediate client peer port. + - ref: network.peer.address + - ref: network.peer.port + - ref: url.path + requirement_level: required + sampling_relevant: true + - ref: url.query + requirement_level: + conditionally_required: If and only if one was received/sent. + sampling_relevant: true + - ref: url.scheme + sampling_relevant: true diff --git a/model/url.yaml b/model/url.yaml new file mode 100644 index 0000000..6e839fc --- /dev/null +++ b/model/url.yaml @@ -0,0 +1,39 @@ +groups: + - id: url + brief: Attributes describing URL. + type: attribute_group + prefix: url + attributes: + - id: scheme + type: string + brief: 'The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol.' + examples: ["https", "ftp", "telnet"] + - id: full + type: string + brief: Absolute URL describing a network resource according to [RFC3986](https://www.rfc-editor.org/rfc/rfc3986) + note: > + For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment + is not transmitted over HTTP, but if it is known, it should be included nevertheless. + + `url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. + In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`. + + `url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) + and SHOULD NOT be validated or modified except for sanitizing purposes. + examples: ['https://www.foo.bar/search?q=OpenTelemetry#SemConv', '//localhost'] + tag: sensitive-information + - id: path + type: string + brief: 'The [URI path](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) component' + examples: ['/search'] + note: When missing, the value is assumed to be `/` + - id: query + type: string + brief: 'The [URI query](https://www.rfc-editor.org/rfc/rfc3986#section-3.4) component' + examples: ["q=OpenTelemetry"] + note: Sensitive content provided in query string SHOULD be scrubbed when instrumentations can identify it. + tag: sensitive-information + - id: fragment + type: string + brief: 'The [URI fragment](https://www.rfc-editor.org/rfc/rfc3986#section-3.5) component' + examples: ["SemConv"] diff --git a/model/user-agent.yaml b/model/user-agent.yaml new file mode 100644 index 0000000..2f43b1a --- /dev/null +++ b/model/user-agent.yaml @@ -0,0 +1,10 @@ +groups: + - id: attributes.user_agent + type: attribute_group + brief: "Describes user-agent attributes." + prefix: user_agent + attributes: + - id: original + type: string + brief: 'Value of the [HTTP User-Agent](https://www.rfc-editor.org/rfc/rfc9110.html#field.user-agent) header sent by the client.' + examples: ['CERN-LineMode/2.15 libwww/2.17b3']