🐛 Bug Report: Traceloop SDK does not always make an API request at the completion of a Colab/Jupyter notebook block

[michael-hhai]

Which component is this bug for?

Traceloop SDK

📜 Description

If you instantiate the Traceloop SDK in Block A:

from traceloop.sdk import Traceloop
import os
# This is just for easy illustrative purposes so people don't need to set up any API keys
os.environ['TRACELOOP_API_KEY'] = 'asdasd'
os.environ['TRACELOOP_BASE_URL'] = 'asdasd'
Traceloop.init(disable_batch=True)

and you run the following code in Block B:

from openai import OpenAI

api_key = input("Please enter your OpenAI API key: ")

client = OpenAI(api_key=api_key)

completion = client.chat.completions.create(
    model='gpt-4o',
    messages=[{"role":"user","content":"What's the meaning of life?"}]
)

print(completion.choices[0].message.content)

then the Traceloop SDK should send an API request at the completion of Block B to record the OpenAI request. With appropriate logging set, you can see that this is not the case.

I see #66, but it seems like this same issue might have reared its head again. I swear this used to work.

👟 Reproduction steps

I have this notebook which reproduces the issue. If you run it, you should see that there is no Traceloop API call in the OpenAI call code block.

👍 Expected behavior

The API call should be triggered at the completion of every code block.

👎 Actual Behavior with Screenshots

The only API calls logged are OpenAI API calls.

🤖 Python Version

No response

📃 Provide any additional context for the Bug.

No response

👀 Have you spent some time to check if this bug has been raised before?

• I checked and didn't find similar issue

Are you willing to submit PR?

None

[nirga]

Hey @michael-hhai! Thanks for this.
Do you mean that you're missing some of the openai spans? Or all of them?

[michael-hhai]

I believe all of them. I think this a regression on #65.

[nirga]

Interesting, since that fix is still there:

openllmetry/packages/traceloop-sdk/traceloop/sdk/tracing/tracing.py

Line 320 in 39f9db6

atexit.register(obj.exit_handler)

I wonder if explicitly calling flush solves this. If so, then we might need to figure out the right signal to listen to within colab to call it ourselves (that is what the line above SHOULD be doing).

[nirga]

@michael-hhai can you try maybe running

from traceloop.sdk.tracing.tracing import TracerWrapper

TracerWrapper().flush()

[michael-hhai]

I did:

from traceloop.sdk import Traceloop
import os
# This is just for easy illustrative purposes so people don't need to set up any API keys
os.environ['TRACELOOP_API_KEY'] = 'asdasd'
os.environ['TRACELOOP_BASE_URL'] = 'asdasd'
Traceloop.init(disable_batch=True)

import http.client as http_client
import logging

http_client.HTTPConnection.debuglevel = 1

logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

from openai import OpenAI

api_key = input("Please enter your OpenAI API key: ")

client = OpenAI(api_key=api_key)

completion = client.chat.completions.create(
    model='gpt-4o',
    messages=[{"role":"user","content":"What's the meaning of life?"}]
)

print(completion.choices[0].message.content)

from traceloop.sdk.tracing.tracing import TracerWrapper

TracerWrapper().flush()

And I got:

Traceback (most recent call last):
  File "/work-folder/venv/lib/python3.12/site-packages/opentelemetry/sdk/trace/export/__init__.py", line 113, in on_end
    self.span_exporter.export((span,))
  File "/work-folder/venv/lib/python3.12/site-packages/opentelemetry/exporter/otlp/proto/grpc/trace_exporter/__init__.py", line 141, in export
    return self._export(spans)
           ^^^^^^^^^^^^^^^^^^^
  File "/work-folder/venv/lib/python3.12/site-packages/opentelemetry/exporter/otlp/proto/grpc/exporter.py", line 262, in _export
    self._client.Export(
  File "/work-folder/venv/lib/python3.12/site-packages/grpc/_channel.py", line 1178, in __call__
    ) = self._blocking(
        ^^^^^^^^^^^^^^^
  File "/work-folder/venv/lib/python3.12/site-packages/grpc/_channel.py", line 1146, in _blocking
    call = self._channel.segregated_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 559, in grpc._cython.cygrpc.Channel.segregated_call
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 428, in grpc._cython.cygrpc._segregated_call
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 422, in grpc._cython.cygrpc._segregated_call
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 262, in grpc._cython.cygrpc._call
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 309, in grpc._cython.cygrpc._call
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 62, in grpc._cython.cygrpc._raise_call_error
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 32, in grpc._cython.cygrpc._call_error
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 23, in grpc._cython.cygrpc._call_error_metadata
TypeError: not all arguments converted during string formatting

[nirga]

That's basically calling an OpenTelemetry method so it might be something with your backend? Are you using GRPC?

[michael-hhai]

Yeah, good call. False alarm on that.

Calling

from traceloop.sdk.tracing.tracing import TracerWrapper

TracerWrapper().flush()

does indeed work.

Would be curious to know why it doesn't always happen upon exit.