OpenTelemetry SDK: Failed to export spans. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180

[swapnil-kotwal-sp]

I've originally posted a query here https://stackoverflow.com/q/76626667/1665592

I'm trying to export the spans to HoneyComb through Otel-Collector

Using below Otel-collector configs I'm able to POST test Trace to HoneyComb UI as below

curl -i http://localhost:4318/v1/traces -X POST -H "Content-Type: application/json"  -d @span.json

My Otel-Collector versions that I tried are as below

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  logging:
   loglevel: debug
  otlphttp:
    endpoint: "https://api.honeycomb.io"
    headers:
      "x-honeycomb-team": "YOUR_API_KEY"
      "x-honeycomb-dataset": "YOUR_DATASET"

processors:
  batch:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, otlphttp]

OR

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  logging:
   loglevel: debug
  otlp:
    endpoint: "http://api.honeycomb.io:443" # even simply api.honeycomb.io:443
    headers:
      "x-honeycomb-team": "YOUR_API_KEY"
      "x-honeycomb-dataset": "YOUR_DATASET"

processors:
  batch:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, otlp]

docker-compose.yaml is as below

version: "2"
services:

  # Collector
  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    restart: always
    command: ["--config=/etc/otel-collector-config.yaml", "${OTELCOL_ARGS}"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - "1888:1888"   # pprof extension
      - "8888:8888"   # Prometheus metrics exposed by the collector
      - "8889:8889"   # Prometheus exporter metrics
      - "13133:13133" # health_check extension
      - "4343:4317"   # OTLP gRPC receiver
      - "4318:4318"   # OTLP http receiver

OpenTelemetry implementation is

Resource resource = Resource.getDefault().merge(Resource.builder().put(SERVICE_NAME, MY_SERVICE_NAME).build());

        OtlpGrpcSpanExporter otlpGrpcSpanExporter = OtlpGrpcSpanExporter.builder().setEndpoint("http://localhost:4318").build();

        SdkTracerProvider sdkTracerProvider = SdkTracerProvider.builder()
                .addSpanProcessor(BatchSpanProcessor.builder(otlpGrpcSpanExporter).build())
                .setResource(resource)
                .build();

        openTelemetry = OpenTelemetrySdk.builder().
                .setTracerProvider(sdkTracerProvider)
                .buildAndRegisterGlobal();

Tracer tracer = openTelemetry.getTracer("My Tracer");

span = tracer.spanBuilder(messageType + " handler span").startSpan();
span.setAttribute("messageType", messageType);

//some logic

span.end();

I'm seeing below error in my application. Not sure what is I'm missing here?

Jul 06, 2023 12:52:04 PM io.opentelemetry.exporter.internal.grpc.GrpcExporterUtil logUnimplemented
SEVERE: Failed to export metrics. Server responded with UNIMPLEMENTED. This usually means that your collector is not configured with an otlp receiver in the "pipelines" section of the configuration. If export is not desired and you are using OpenTelemetry autoconfiguration or the javaagent, disable export by setting OTEL_METRICS_EXPORTER=none. Full error message: unknown service opentelemetry.proto.collector.metrics.v1.MetricsService



Jul 06, 2023 12:52:08 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
SEVERE: Failed to export spans. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180

Also, Please note, When I post using 4317 gRPC port I got below error.

curl -i http://0.0.0.0:4317/v1/traces -X POST -H "Content-Type: application/json"  -d @span.json

throws
curl: (1) Received HTTP/0.9 when not allowed

[swapnil-kotwal-sp]

I think I was pointing to wrong port no. i.e. 4317 (which is internal container port but external port was 4343 that I've mentioned in docker-compose.yaml

OtlpGrpcSpanExporter otlpGrpcSpanExporter = 
OtlpGrpcSpanExporter.builder().setEndpoint("http://localhost:4343").build();

P.S.: http protocol was working previously as well.