Opentelemetry javaagent

[sarkisjad]

Hello,

I am attaching the opentelemetry-javaagent to a JVM.
The problem I’m facing is: when the JVM is finished executing, the javaagent will keep on looking to export metrics to an exporter if it does not find any.
If it ends up not finding any exporter, it will cause the JVM to hang until it does.
How can I tell the javaagent to abort when the main JVM is done executing?

Thanks,
Jad

[trask]

Hi @sarkisjad, this is unexpected behavior, can you open an issue and submit a repro that can be used to troubleshoot the problem?

[sarkisjad]

Hello @trask,

To give you more context, here's my use case:

I am using ConfigurableMetricExporterProvider SPI to provide a metric exporter implementation and setting the otel.metrics.exporter configuration to my chosen exporter name.
I am also packaging my exporter in the java agent extension and setting the collection interval otel.metric.export.interval to 10 seconds.

What is happening is, when the main JVM (that has the javaagent attached to it) is finished, the exporter is still trying to export metrics every 10 seconds. This causes the JVM to hang indefinitely.

Is there something we can do on the side of the extension to tell it to abort whenever the main JVM is finished?

Thanks

[jkwatson]

I suspect that either a) you're creating a thread that is not a daemon thread, thus causing the JVM to not be able to exit, or b) you're not properly shutting down your exporter (or some combination of the two). I would check these things first.

[sarkisjad]

Hello,

I am also using the injected java auto-instrumentation agent to monitor the desired JMX MBeans like is required and explained here.
Do you think this has something to do with the thread not terminating when the JVM is done?

I am asking because the exporter I created is very basic and does not create a new thread. here's the exporter I created:

@AutoService(ConfigurableMetricExporterProvider.class)
public class CustomExporter implements ConfigurableMetricExporterProvider{

    @Override
    public MetricExporter createExporter(ConfigProperties configProperties) {
        return new MyOwnExporter();
    }

    @Override
    public String getName() {
        return "my_exporter";
    }

    private static class MyOwnExporter implements MetricExporter{

        @Override
        public CompletableResultCode export(Collection<MetricData> collection) {
            System.out.println("export");
            return CompletableResultCode.ofFailure();
        }

        @Override
        public CompletableResultCode flush() {
            return CompletableResultCode.ofFailure();
        }

        @Override
        public CompletableResultCode shutdown() {
            System.out.println("shutdown");
            return CompletableResultCode.ofFailure();
        }

        @Override
        public AggregationTemporality getAggregationTemporality(InstrumentType instrumentType) {
            return AggregationTemporality.CUMULATIVE;
        }
    }
}

And here's the rules file I added (Note that when I remove the property -Dotel.jmx.config, the problem is solved which is the reason why I think the issue is coming from there):

rules:
  - bean: java.lang:name=*,type=GarbageCollector
    metricAttribute:
      generation: param(name)
    prefix: java.
    mapping:
      LastGcInfo.duration:
        metric: duration
        desc: The duration of the last GC.
        type: gauge
        unit: ms

  - bean: java.lang:type=Memory
    mapping:
      HeapMemoryUsage.max:
        metric: memory.limit
        type: gauge
        desc: The maximum amount of memory a process can use 
        unit: By

Do you th ink that might have something to do with it?

Thanks,
Jad

[jkwatson]

I honestly don't know what this config is about, or how it gets used by the Javaagent (@trask might have more info here).

I'm curious why you had all of the methods (flush, shutdown, etc) return failures, rather than success. I don't know if that could impact things, but it seems strange to consider a no-op a failure. I would definitely recommend making those methods return CompletableResultCode.ofSuccess() even if that is unrelated to this issue.