Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved Logging and Header Propagation in Orka #339

Merged
merged 12 commits into from
Oct 4, 2023
Merged

Improved Logging and Header Propagation in Orka #339

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
b239efb
Change orka's default requestIds
nikostoulas Sep 27, 2023
738bd1b
Riviere: do not overwrite user provided headers
nikostoulas Sep 28, 2023
189ff63
Update rabbitmq example code
nikostoulas Oct 2, 2023
a74d735
Install chalk
nikostoulas Oct 2, 2023
4b2c077
Deprecate alsSupported usage
nikostoulas Oct 2, 2023
4125293
Introduce propagatedHeaders and utils to manage them
nikostoulas Oct 2, 2023
81801b6
Propagate headers for RabbitMQ handlers
nikostoulas Oct 2, 2023
1f468a5
Improve non json logs
nikostoulas Oct 2, 2023
91f9d27
Update docs
nikostoulas Oct 2, 2023
3b40e03
Add/fix tests
nikostoulas Oct 3, 2023
9d886f8
fixup! Propagate headers for RabbitMQ handlers
nikostoulas Oct 3, 2023
47509a7
Add traceHeader to propagatedHeaders by default
nikostoulas Oct 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 63 additions & 70 deletions __snapshots__/diamorphosis.test.ts.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ exports['Diamorphosis Test should set json/console loggingvariables when nothing
},
"printLogo": true,
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "debug",
"console": true,
"json": false
Expand Down Expand Up @@ -192,32 +192,31 @@ exports['Diamorphosis Test should set json/console loggingvariables when nothing
"logKeys": [
"requestId",
"visitor",
"correlationId"
"correlationId",
"propagatedHeaders"
],
"istioTraceContextHeaders": {
"propagatedHeaders": {
"enabled": true,
"headers": [
"cf-ray",
"x-request-id",
"x-b3-traceid",
"x-b3-spanid",
"x-b3-parentspanid",
"x-b3-sampled",
"x-b3-flags",
"x-ot-span-context"
]
},
"headerPropagation": {
"enabled": true,
"headers": [
"cf-ray"
"x-ot-span-context",
"x-depth",
"x-parent-id",
"x-initiator-id"
]
}
}
}

exports['Diamorphosis Test should set json/console loggingvariables when console:not set, json:true, styles:[] shoud be console:false, json:true, styles:["json"] 1'] = {
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "debug",
"console": false,
"json": true
Expand Down Expand Up @@ -411,32 +410,31 @@ exports['Diamorphosis Test should set json/console loggingvariables when console
"logKeys": [
"requestId",
"visitor",
"correlationId"
"correlationId",
"propagatedHeaders"
],
"istioTraceContextHeaders": {
"propagatedHeaders": {
"enabled": true,
"headers": [
"cf-ray",
"x-request-id",
"x-b3-traceid",
"x-b3-spanid",
"x-b3-parentspanid",
"x-b3-sampled",
"x-b3-flags",
"x-ot-span-context"
]
},
"headerPropagation": {
"enabled": true,
"headers": [
"cf-ray"
"x-ot-span-context",
"x-depth",
"x-parent-id",
"x-initiator-id"
]
}
}
}

exports['Diamorphosis Test should set json/console loggingvariables when console:true, json:true, styles:[] shoud be console:true, json:true, styles:["json"] 1'] = {
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "debug",
"console": true,
"json": true
Expand Down Expand Up @@ -630,32 +628,31 @@ exports['Diamorphosis Test should set json/console loggingvariables when console
"logKeys": [
"requestId",
"visitor",
"correlationId"
"correlationId",
"propagatedHeaders"
],
"istioTraceContextHeaders": {
"propagatedHeaders": {
"enabled": true,
"headers": [
"cf-ray",
"x-request-id",
"x-b3-traceid",
"x-b3-spanid",
"x-b3-parentspanid",
"x-b3-sampled",
"x-b3-flags",
"x-ot-span-context"
]
},
"headerPropagation": {
"enabled": true,
"headers": [
"cf-ray"
"x-ot-span-context",
"x-depth",
"x-parent-id",
"x-initiator-id"
]
}
}
}

exports['Diamorphosis Test should set json/console loggingvariables when console:false, json:true, styles:[] shoud be console:false, json:true, styles:["json"] 1'] = {
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "debug",
"console": false,
"json": true
Expand Down Expand Up @@ -849,32 +846,31 @@ exports['Diamorphosis Test should set json/console loggingvariables when console
"logKeys": [
"requestId",
"visitor",
"correlationId"
"correlationId",
"propagatedHeaders"
],
"istioTraceContextHeaders": {
"propagatedHeaders": {
"enabled": true,
"headers": [
"cf-ray",
"x-request-id",
"x-b3-traceid",
"x-b3-spanid",
"x-b3-parentspanid",
"x-b3-sampled",
"x-b3-flags",
"x-ot-span-context"
]
},
"headerPropagation": {
"enabled": true,
"headers": [
"cf-ray"
"x-ot-span-context",
"x-depth",
"x-parent-id",
"x-initiator-id"
]
}
}
}

exports['Diamorphosis Test should set json/console loggingvariables when console:not set, json:true, styles:["simple"] shoud be console:false, json:true, styles:["simple"] 1'] = {
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "debug",
"console": false,
"json": true
Expand Down Expand Up @@ -1068,24 +1064,23 @@ exports['Diamorphosis Test should set json/console loggingvariables when console
"logKeys": [
"requestId",
"visitor",
"correlationId"
"correlationId",
"propagatedHeaders"
],
"istioTraceContextHeaders": {
"propagatedHeaders": {
"enabled": true,
"headers": [
"cf-ray",
"x-request-id",
"x-b3-traceid",
"x-b3-spanid",
"x-b3-parentspanid",
"x-b3-sampled",
"x-b3-flags",
"x-ot-span-context"
]
},
"headerPropagation": {
"enabled": true,
"headers": [
"cf-ray"
"x-ot-span-context",
"x-depth",
"x-parent-id",
"x-initiator-id"
]
}
}
Expand Down Expand Up @@ -1213,7 +1208,7 @@ exports['Diamorphosis Test should set json/console loggingvariables when console
},
"printLogo": false,
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "fatal",
"console": true,
"json": false
Expand Down Expand Up @@ -1292,24 +1287,23 @@ exports['Diamorphosis Test should set json/console loggingvariables when console
"logKeys": [
"requestId",
"visitor",
"correlationId"
"correlationId",
"propagatedHeaders"
],
"istioTraceContextHeaders": {
"propagatedHeaders": {
"enabled": true,
"headers": [
"cf-ray",
"x-request-id",
"x-b3-traceid",
"x-b3-spanid",
"x-b3-parentspanid",
"x-b3-sampled",
"x-b3-flags",
"x-ot-span-context"
]
},
"headerPropagation": {
"enabled": true,
"headers": [
"cf-ray"
"x-ot-span-context",
"x-depth",
"x-parent-id",
"x-initiator-id"
]
}
}
Expand Down Expand Up @@ -1437,7 +1431,7 @@ exports['Diamorphosis Test should set json/console loggingvariables when console
},
"printLogo": false,
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "fatal",
"console": true,
"json": true
Expand Down Expand Up @@ -1518,24 +1512,23 @@ exports['Diamorphosis Test should set json/console loggingvariables when console
"logKeys": [
"requestId",
"visitor",
"correlationId"
"correlationId",
"propagatedHeaders"
],
"istioTraceContextHeaders": {
"propagatedHeaders": {
"enabled": true,
"headers": [
"cf-ray",
"x-request-id",
"x-b3-traceid",
"x-b3-spanid",
"x-b3-parentspanid",
"x-b3-sampled",
"x-b3-flags",
"x-ot-span-context"
]
},
"headerPropagation": {
"enabled": true,
"headers": [
"cf-ray"
"x-ot-span-context",
"x-depth",
"x-parent-id",
"x-initiator-id"
]
}
}
Expand Down
5 changes: 3 additions & 2 deletions docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ You can find the default values below:
},
"printLogo": true,
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "debug",
"console": true,
"json": false
Expand Down Expand Up @@ -255,9 +255,10 @@ You can find the default values below:
"requestContext": {
"enabled": true,
"logKeys": ["requestId", "visitor"],
"istioTraceContextHeaders": {
"propagatedHeaders": {
"enabled": true,
"headers": [
"cf-ray",
"x-request-id",
"x-b3-traceid",
"x-b3-spanid",
Expand Down
2 changes: 1 addition & 1 deletion docs/logs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ By default the below config is used:
```json
{
"log": {
"pattern": "%[[%d] [%p] %c%] %x{logTracer} %m",
"pattern": "%[[%d] [%p] %c%] %x{requestId}%m %x{logTracer}",
"level": "debug",
"console": true,
"json": false
Expand Down
42 changes: 18 additions & 24 deletions docs/request-context.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,34 +20,32 @@ Orka by default exposes a Request Context to the app.
It can be used for setting/getting variables that can be accessible in every code file (within the request's life). It uses nodeJS `async_hooks` to store this information.

By default it appends in the request context: `requestId`, `visitor` (if you have the option `config.visitor.cookie`), `ddSpan`(if you have DataDog enabled),
and `istio-headers` that holds [Istio trace context headers](https://istio.io/latest/docs/tasks/observability/distributed-tracing/overview/#trace-context-propagation) (if you have the option `config.requestContext.istioTraceContextHeaders.enabled`).
and `propagatedHeaders` that holds [Istio trace context headers](https://istio.io/latest/docs/tasks/observability/distributed-tracing/overview/#trace-context-propagation) (if you have the option `config.requestContextl.enabled`).
The `requestId` is retrieved using the option `config.traceHeaderName`

### Log Tracer

If the Request Context is enabled, orka by default appends `requestId`, `correlationId` and `visitor` to all the application logs automatically.

### Istio trace context headers
### Header Propagation

Istio trace context headers are enabled by default as long as the current nodejs version supports AsyncLocalStorage (async_hooks).
Header propagation enabled by default as long as the current nodejs version supports AsyncLocalStorage (async_hooks).

If option `config.requestContext.istioTraceContextHeaders` is enabled, orka propagates [Istio trace context headers](https://istio.io/latest/docs/tasks/observability/distributed-tracing/overview/#trace-context-propagation)
If option `config.requestContext.propagatedHeaders` is enabled, orka propagates some [Istio trace context headers](https://istio.io/latest/docs/tasks/observability/distributed-tracing/overview/#trace-context-propagation) and cloudflare `cf-ray`
from incoming http requests to any outbound http request.

By default orka propagates the following Istio headers:
`"x-request-id", "x-b3-traceid", "x-b3-spanid", "x-b3-parentspanid", "x-b3-sampled", "x-b3-flags", "x-ot-span-context"`

This list can be modified by changing `config.requestContext.istioTraceContextHeaders.headers`.

### Header Propagation
Cloudflare headers are appended to the context by default as long as the current nodejs version supports AsyncLocalStorage (async_hooks).

If option `config.request.headerPropagation` is enabled, orka propagates any header specified in its whitelist.

By default the following headers are propagated:
- cf-ray

The whitelist can be overriden by changing `config.requestContext.headerPropagation.headers` value.
- "cf-ray"
- "x-request-id"
- "x-b3-traceid"
- "x-b3-spanid"
- "x-b3-parentspanid"
- "x-b3-sampled"
- "x-b3-flags"
- "x-ot-span-context"
- "x-depth"

This list can be modified by changing `config.requestContext.propagatedHeaders.headers`.

### Configuration

Expand All @@ -57,14 +55,10 @@ If you don't specify anything in your `config.requestContext` it defaults to:
{
"requestContext": {
"enabled": true,
"logKeys": ["requestId", "visitor", "correlationId"], // These are the keys that will be appended automatically to your logs
"istioTraceContextHeaders": {
"enabled": true,
"headers": ["x-request-id", "x-b3-traceid", "x-b3-spanid", "x-b3-parentspanid", "x-b3-sampled", "x-b3-flags", "x-ot-span-context"]
},
"headerPropagation": {
"logKeys": ["requestId", "visitor", "correlationId", "propagatedHeaders"], // These are the keys that will be appended automatically to your logs
"propagatedHeaders": {
"enabled": true,
"headers": ["cf-ray"]
"headers": ["cf-ray", "x-request-id", "x-b3-traceid", "x-b3-spanid", "x-b3-parentspanid", "x-b3-sampled", "x-b3-flags", "x-ot-span-context"]
}
}
}
Expand Down
Loading
Loading