Define compatibility between cloud events and OpenTelemetry events

[jack-berg]

CloudEvents is a popular standard for describing event data. OpenTelemetry defines an Event API and has an active Event working group to resolve some of the outstanding issues related to the Event API / SDK and Events in semantic conventions.

Similar to the compatibility documents we provide for Prometheus / OpenMetrics, OpenCensus, and OpenTracing, we should provide a compatibility document describing how CloudEvents interoperate with OpenTelemetry Events.

We should be careful to ensure that any work done with the Event API / SDK and Events in the semantic conventions does not prevent interoperability with CloudEvents.

Additional context:

• Consider deleting Events API and recommend CloudEvents API with a bridge to Otel Log SDK #2999: We considered using CloudEvents API instead of introducing an OpenTelemetry Events API. The idea was ultimately rejected.
• Events interoperation with CloudEvents #319: A long standing request for CloudEvent interoperation with OpenTelemetry. I believe we need a separate issue than this because I'm proposing specifics of how that interoperation works: A compatibility document which describes a mapping between CloudlEvents and OpenTelemetry Events.
• Add semantic conventions for CloudEvents #1949: Introduced semantic conventions describing how to model CloudEvent messages as spans.

[pyohannes]

There are already semantic conventions for tracing the flow of CloudEvents, which are build on a Distributed Tracing extension that is part of CloudEvents.

It might be necessary to consider (and maybe re-work) this when defining a consistent relationship between OpenTelemetry events and CloudEvents.

[tedsuo]

I suggest that this issue be moved to semantic conventions. There are no compatibility issues.

The difficult design work involved collaborating with CloudEvents to design tracing for messaging systems. Structurally, CloudEvents are flat. All CloudEvents metadata can be mapped directly to attributes using the cloudevent namespace. CloudEvent data maps directly to the payload. There are a handful of CloudEvent attributes which have not yet been added to semantic conventions, but I do not see any additional design work as necessary at this time. CloudEvents which do not have a tracing context must be stored as events, but otherwise they are no different than CloudEvents stored as spans.

There is one important note. While it is straightforward for OpenTelemetry receive CloudEvents, it is not guaranteed that all telemetry received by OpenTelemetry can be converted into CloudEvents. OpenTelemetry intends to collect the widest range of telemetry available. OpenTelemetry is not intended to function as an application component that produces CloudEvents as part of the data plane of a distributed system. In practice, this means that OpenTelemetry can accept data which may be missing information required by the CloudEvents specification. Specifically, we do not require that end users to provide a unique id, a source URI-reference, a CloudEvents specversion, or a type.

[tedsuo]

One other subject that should be addressed is integration at the instrumentation level. CloudEvents provides SDKs in many languages, which are how they expect CloudEvent users to interact with the project. I don't think there are any plans to merge the projects and retire the CloudEvents SDKs, there is much more functionality in the CloudEvents SDKs than in OTel's simple API for sending events as telemetry,

For these reasons, I don't believe that we should add any CloudEvent functionality to the Otel API, or otherwise encourage CloudEvent users to abandon the CloudEvent SDKs in favor of the Otel SDKs. They are much better equipped to keep up with the needs of their project than we are.

Integration between OTel and CloudEvents could be as simple as the CloudEvents SDK calling the Otel Event API under the hood. When Otel is not installed, the API call is a no-op. When OTel is installed, CloudEvents will automatically be integrated with any OTel instrumentation that is running in the application. That should be sufficient.

[pyohannes]

For these reasons, I don't believe that we should add any CloudEvent functionality to the Otel API, or otherwise encourage CloudEvent users to abandon the CloudEvent SDKs in favor of the Otel SDKs.

💯 The scope of CloudEvents is much larger than that of OTel events. While the latter are restricted to telemetry, CloudEvents aims at supporting any workflows that involve events, see here:

CloudEvents is a specification for describing event data in common formats to provide interoperability across services, platforms and systems.

I assume we don't want OTel Event APIs to emit events that are used to control some business logic.

[tigrannajaryan]

@tedsuo @MSNev do you want to keep this open? If so let me know who is going to the assignee.

[MSNev]

We just decided that from the "event WG" perspective that this was not in scope (which is why we removed it from our board).

As the definition of a specific "event" would be for the event domain "owners" to specify in this case that would be someone who wants to create a bridge from Cloud Events to OTel. So we don't have anyone specific to be the assignee.

[tigrannajaryan]

Can we close this issue?