Hi @ptanov 👋

Thanks for opening the issue 🙌

Gotcha, I think this is a solid motivation. I might find some time to implement it. But could you help me a bit on the road with understanding the traceparent header.

So a request comes in with a header named traceparent and then we should set this in the span in TGI? Do I understand it correctly?

@ErikKaum yeah essentially, most frameworks do this automatically and I am surprised the axum opentelemetry integration doesn't just do this automatically.

Specifically, you usually check to see if there is an "active trace" before starting a new one, so wherever the integration is starting a new trace/determining if there is an active trace is where you'd probably need to look to see if it even knows about active traces.

Ideally, there is some middleware layer that parses out the trace from the header and sets the "active trace" immediately.

Hi @ErikKaum ,
sorry for the late reply :(

Yes, it is exactly as described by @noyoshi - it is usually done automatically by the framework.

The format of the content of the header is described here: https://www.w3.org/TR/trace-context/#traceparent-header

No worries @ptanov, it's been quite a busy week so (un)fortunately I haven't gotten to this yet 😅

Thanks for the clarification @ptanov 🙌 A bit surprising indeed!

I think I won't be able to get to this this week, but most likely end of next week.

Thank you!

One more thing that could be useful to you- usually if trace is not automatically propagated by the framework (e.g. by .layer(OtelInResponseLayer::default()) and .layer(OtelAxumLayer::default()) in ./router/src/server.rs) it could be extracted and reused manually with something like:

    let extractor = HeaderExtractor(headers);
    let propagator = global::get_text_map_propagator(|propagator| propagator.clone());
    let span = propagator.extract(&extractor);
    let _guard = span.make_current();

assuming that global::set_text_map_propagator(TraceContextPropagator::new()); is set in ./router/src/logging.rs

but I couldn't make it work :( Sorry I'm not familiar with TGI codebase nor with rust.

I'm testing it by providing tracepoint header to curl, e.g. curl -v tgihostport/v1/chat/completions -H 'traceparent: 00-aa1274dfd3752a0f67b3bc3f21c71aaa-c8480007d4f1cbbb-01' -H 'Content-Type: application/json' -d '{SOMEQUESTION}'

Thank you @ptanov 🚀 this already helps a lot!

Hi @ErikKaum,
can I help you with something regarding this ticket (e.g. docker-compose configuration for running collector/kibana/prometheus)?

Hi @ptanov

Sorry for the radio silence, unfortunately it's been very busy lately and I haven't been able to prioritize this 🙁

Hi @ErikKaum ,
don't worry, take your time

73 ;)

@ptanov, @ErikKaum

Hi, I just opened a PR for this feature in the TEI repo here! If you are available, please check it out and feel free to leave a review!

I'm new to OpenTelemetry things, so maybe my implementation could be incorrect or inefficient.

Thanks @kozistr ,
I'll try to check it tomorrow

Hi @kozistr @ErikKaum ,

I tested this PR and I can confirm that it works in my case - opentelemetry trace is propagated from request to TGI.

Thanks again @kozistr !

Big thanks @kozistr 🙌

Hi @kozistr @ErikKaum ,

I tested this PR and I can confirm that it works in my case - opentelemetry trace is propagated from request to TGI.

Thanks again @kozistr !

great to hear! thanks for tesing this PR :)