Clarify the mapping between Google Cloud Logging and OpenTelemetry #3775
Conversation
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
|
I'm waiting for feedback on the linked issue (as described in the PR process). We can have a conversation here if that works better. |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
* Clarify that the JSON representation is used for the LogEntry * Change to Pascal-case for representing the field in LogEntry * Map MonitoredResource to resource attributes with gcp prefix * Clarify the type of the Body for jsonPayload, protoPayload, and textPayload * Mapping section for Severity from Cloud Logging to OpenTelemetry * Mapping section for Severity from OpenTelemetry to Cloud Logging * Semantic Mapping section (only includes log.record.uid) * Clarified that the JSON representation is used for the LogEntry * Map LogEntryOperation to resource attribute with gcp.operation (KVList) * Map LogEntrySourceLocation to resource attribute with gcp.source_location (KVList) * Map LogSplit to resource attribute with gcp.log_split (KVList) With this [draft collector PR](open-telemetry/opentelemetry-collector-contrib#29299), the PubsubReceiver will be able to handle the native translation of Google Cloud LogEntry's. But translations are opinionated. I prefer that we agree on the target mapping before merging the PR.
|
I'm pushing the implementation in the collector, so it makes sense to have a compatibility section. I've marked the implementation as |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
|
not stale... waiting till open-telemetry/opentelemetry-collector-contrib#29299 gets merged... |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
|
Closed as inactive. Feel free to reopen if this PR is still being worked on. |
There was a problem hiding this comment.
@alexvanboxel unsure if you plan to keep this moving, but I added a couple of comments just in case
| | `resource` | `MonitoredResource` | The monitored resource that produced this log entry. | `Resource["gcp.*"]` | | ||
| | `httpRequest` | `HttpRequest` | The HTTP request associated with the log entry, if any. | `Attributes["gcp.http_request"]` (KVList) | | ||
| | `operation` | `LogEntryOperation` | Information about a operation associated with the log entry. | `Attributes["gcp.operation"]` (KVList) | | ||
| | `sourceLocation` | `LogEntrySourceLocation` | Source code location information associated with the log entry. | `Attributes["gcp.source_location"]` (KVList) | |
There was a problem hiding this comment.
looking at docs for LogEntrySourceLocation I this ought to be mapped to the equivalent semconvcode.* attributes as follows:
sourceLocation.file=>code.filepathsourceLocation.line=>code.lineno
Less sure of what to do with sourceLocation.function, which is described as
Human-readable name of the function or method being invoked, with optional context such as the class or package name. This information may be used in contexts such as the logs viewer, where a file and line number are less meaningful. The format can vary by language. For example: qual.if.ied.Class.method (Java), dir/package.func (Go), function (Python).
it ought to go into the code.namespace and code.function attributes. if you happen to have a sample of data across different languages for which gcp has a logging integration, perhaps the right thing to do can be figured out based on that? failing that, a reasonable best effort might be (python style split / indexing):
code.functiongetssourceLocation.function.split(".")[-1]code.namespacegetssourceLocation.function.split(".")[:-1]
| | `traceSampled` | `boolean` | The sampling decision of the trace associated with the log entry. | `TraceFlags.SAMPLED` | | ||
| | `labels` | `map<string,string>` | A set of user-defined (key, value) data that provides additional information about the log entry. | `Attributes` | | ||
| | `resource` | `MonitoredResource` | The monitored resource that produced this log entry. | `Resource["gcp.*"]` | | ||
| | `httpRequest` | `HttpRequest` | The HTTP request associated with the log entry, if any. | `Attributes["gcp.http_request"]` (KVList) | |
There was a problem hiding this comment.
could be worth attempting to map these to http semconv? not sure why I didn't do it initially.
requestMethod=>http.request.methodrequestUrl=>url.fullrequestSize=> see [0] belowstatus=>http.response.status_coderesponseSize=> see [0] belowuserAgent=>user_agent.originalremoteIp=>client.addressserverIp=>server.addressreferer=>http.request.header.referer[0]protocol=>net.protocol.name(lower case),net.protocol.version
The remaining ones can be mapped to gcp.*:
latencycacheLookupcacheHitcacheValidatedWithOriginServercacheFillBytes
[0] I had previously opened open-telemetry/semantic-conventions#84 to add "whole-request" size attributes, ie headers + body. I will rebase / update the PR this week to see if it gets any traction.
Changed
Added
Why
With this draft collector PR, the PubsubReceiver will be able to handle the native translation of Google Cloud LogEntry's. But translations are opinionated. I prefer that we agree on the target mapping before merging the PR.
Fixes #
Related issues:
#3774
open-telemetry/opentelemetry-collector-contrib#23184
Draft PR in the Collector:
open-telemetry/opentelemetry-collector-contrib#29299