Native Image Build Fails after upgrading dd-trace-java from 1.40.0 to 1.43.0

[mimiyanwk]

I am attempting to build a native image for my quarkus application as described on the website https://docs.datadoghq.com/tracing/trace_collection/compatibility/java/?tab=quarkusnative, but I am encountering initialization issues related to classes during the build time. These issues have arisen after upgrading Datadog's Java APM agent from version 1.40.0 to 1.43.0.

When building the native image, I receive the following error messages:

The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=. Or you can write your own initialization methods and call them explicitly from your main entry point.

Object was reached by reading field datadog.trace.util.AgentTaskScheduler.worker of constant 
datadog.trace.util.AgentTaskScheduler@7fb566f7: datadog.trace.util.AgentTaskScheduler@7fb566f7
  scanning root datadog.trace.util.AgentTaskScheduler@7fb566f7: datadog.trace.util.AgentTaskScheduler@7fb566f7 embedded in
    datadog.trace.agent.common.metrics.OkHttpSink.accept(OkHttpSink.java:85)
  parsing method datadog.trace.agent.common.metrics.OkHttpSink.accept(OkHttpSink.java:74) reachable via the parsing context
    at datadog.trace.agent.core.datastreams.MsgPackDatastreamsPayloadWriter.writePayload(MsgPackDatastreamsPayloadWriter.java:116)
    at datadog.trace.agent.core.datastreams.DefaultDataStreamsMonitoring.flush(DefaultDataStreamsMonitoring.java:417)
    at datadog.trace.agent.core.datastreams.DefaultDataStreamsMonitoring.access$900(DefaultDataStreamsMonitoring.java:52)
    at datadog.trace.agent.core.datastreams.DefaultDataStreamsMonitoring$InboxProcessor.run(DefaultDataStreamsMonitoring.java:363)
    at java.util.OptionalInt.ifPresentOrElse(OptionalInt.java:185)
    at root method.(Unknown Source)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.heap.ImageHeapScanner.onObjectReachable(ImageHeapScanner.java:591)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.heap.SVMImageHeapScanner.onObjectReachable(SVMImageHeapScanner.java:125)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.heap.ImageHeapScanner.lambda$markReachable$10(ImageHeapScanner.java:572)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:166)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:152)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute(ForkJoinTask.java:1726)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute(ForkJoinTask.java:1717)
        at java.base/java.util.concurrent.ForkJoinTask$InterruptibleTask.exec(ForkJoinTask.java:1641)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
        at java.base/java.util.concurrent.ForkJoinPool.externalHelpQuiesce(ForkJoinPool.java:2535)
        at java.base/java.util.concurrent.ForkJoinPool.helpQuiescePool(ForkJoinPool.java:2569)
        at java.base/java.util.concurrent.ForkJoinPool.awaitQuiescence(ForkJoinPool.java:3808)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.complete(CompletionExecutor.java:208)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.heap.HeapSnapshotVerifier.checkHeapSnapshot(HeapSnapshotVerifier.java:105)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.heap.SVMImageHeapVerifier.checkHeapSnapshot(SVMImageHeapVerifier.java:51)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.AbstractAnalysisEngine.analysisModified(AbstractAnalysisEngine.java:217)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.AbstractAnalysisEngine.runAnalysis(AbstractAnalysisEngine.java:190)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:832)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:592)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:554)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:528)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:711)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:139)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:94)

Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected a started Thread in the image heap. Thread name: dd-task-scheduler. Threads running in the image generator are no longer running at image runtime. If these objects should not be stored in the image heap, you can use

'--trace-object-instantiation=java.lang.Thread'

to find classes that instantiate these objects. Once you found such a class, you can mark it explicitly for run time initialization with

'--initialize-at-run-time=<culprit>'

to prevent the instantiation of the object.

The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=. Or you can write your own initialization methods and call them explicitly from your main entry point.

    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.error(DisallowedImageHeapObjectFeature.java:186)
    at org.graalvm.nativeimage.builder/com.oracle.svm.core.image.DisallowedImageHeapObjects.onThreadReachable(DisallowedImageHeapObjects.java:124)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.lambda$duringSetup$4(DisallowedImageHeapObjectFeature.java:79)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisType.lambda$notifyObjectReachable$12(AnalysisType.java:659)
    at java.base/java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4714)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.ConcurrentLightHashSet.forEach(ConcurrentLightHashSet.java:146)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisType.notifyObjectReachable(AnalysisType.java:659)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.heap.ImageHeapScanner.onObjectReachable(ImageHeapScanner.java:586)
    ... 23 more

I build the quarkus native using:
Quarkus Version: 3.15
Builder Image: quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-23

Command:

 ./mvnw package -Dnative -Dquarkus.native.additional-build-args='-J-javaagent:/datadog-lib/dd-java-agent.jar,-J-Ddd.profiling.enabled=true,--enable-monitoring=jfr'

Workaround
I must mark the classes explicitly for run time initialization with:

./mvnw package -Dnative -Dquarkus.native.additional-build-args='-J-javaagent:/datadog-lib/dd-java-agent.jar,-J-Ddd.profiling.enabled=true,--enable-monitoring=jfr,--initialize-at-run-time=datadog.trace.agent.core.scopemanager.ContinuableScopeManager\,datadog.trace.bootstrap.instrumentation.java.concurrent.QueueTimerHelper$RateLimiterHolder'

[mcculls]

Hi @mimiyanwk - thanks for the additional details

Do you see the same error messages when using quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21 ?

[mimiyanwk]

Yes, I have seen the same error message.
The workaround works also for quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21.
There is no error when using the version 1.40.0 and quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21.