Otel Java Instrumentation not working in JDK23

[nkvignesh]

Describe the bug
AWS Otel java instrumentation v1.32.5 looks to be not compatible with JDK23. OpenTelemetry Javaagent failed to start.

Steps to reproduce
Test Application

public class TestApp {
  public static void main(String[] args) {
    System.out.println("TestApp");
  }
}

To Run Open Telemetry

javac TestApp.java
java -javaagent:otel-agent.jar TestApp

What did you expect to see?
OpenTelemetry agent up & running without any initialization issue.

Expected Output

OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[otel.javaagent 2024-11-14 10:24:55:492 +0000] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 1.32.5-aws
TestApp
[otel.javaagent 2024-11-14 10:25:01:117 +0000] [OkHttp http://localhost:4317/...] WARN io.opentelemetry.exporter.internal.grpc.GrpcExporter - Failed to export metrics. Server responded with gRPC status code 2. Error message: Failed to connect to localhost/127.0.0.1:4317

What did you see instead?
OpenTelemetry agent failed to start

OpenTelemetry Javaagent failed to start
java.lang.ExceptionInInitializerError
        at io.opentelemetry.javaagent.tooling.muzzle.AgentTooling.<clinit>(AgentTooling.java:23)
        at io.opentelemetry.javaagent.tooling.AgentInstaller.installBytebuddyAgent(AgentInstaller.java:151)
        at io.opentelemetry.javaagent.tooling.AgentInstaller.installBytebuddyAgent(AgentInstaller.java:103)
        at io.opentelemetry.javaagent.tooling.AgentStarterImpl.start(AgentStarterImpl.java:98)
        at io.opentelemetry.javaagent.bootstrap.AgentInitializer$1.run(AgentInitializer.java:53)
        at io.opentelemetry.javaagent.bootstrap.AgentInitializer$1.run(AgentInitializer.java:47)
        at io.opentelemetry.javaagent.bootstrap.AgentInitializer.execute(AgentInitializer.java:64)
        at io.opentelemetry.javaagent.bootstrap.AgentInitializer.initialize(AgentInitializer.java:46)
        at io.opentelemetry.javaagent.OpenTelemetryAgent.startAgent(OpenTelemetryAgent.java:57)
        at io.opentelemetry.javaagent.OpenTelemetryAgent.agentmain(OpenTelemetryAgent.java:49)
        at software.amazon.opentelemetry.javaagent.bootstrap.AwsAgentBootstrap.agentmain(AwsAgentBootstrap.java:28)
        at software.amazon.opentelemetry.javaagent.bootstrap.AwsAgentBootstrap.premain(AwsAgentBootstrap.java:24)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:560)
        at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:572)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.findLoadedClass(java.lang.String) accessible: module java.base does not "opens java.lang" to unnamed module @6b71769e
        at java.base/java.lang.reflect.AccessibleObject.throwInaccessibleObjectException(AccessibleObject.java:388)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:364)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:312)
        at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:203)
        at java.base/java.lang.reflect.Method.setAccessible(Method.java:197)
        at io.opentelemetry.javaagent.tooling.muzzle.AgentCachingPoolStrategy.getFindLoadedClassMethod(AgentCachingPoolStrategy.java:115)
        at io.opentelemetry.javaagent.tooling.muzzle.AgentCachingPoolStrategy.<clinit>(AgentCachingPoolStrategy.java:58)
        ... 16 more

Additional context
If I apply the workaround, it works. But It will be unlikely that the workaround will be accepted in the production code base.

Workaround
export JDK_JAVA_OPTIONS="--add-opens java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED"

sh-5.2# java --version
NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED
openjdk 23.0.1 2024-10-15
OpenJDK Runtime Environment Corretto-23.0.1.8.1 (build 23.0.1+8-FR)
OpenJDK 64-Bit Server VM Corretto-23.0.1.8.1 (build 23.0.1+8-FR, mixed mode, sharing)
sh-5.2# export JDK_JAVA_OPTIONS="--add-opens java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED"
sh-5.2# java -javaagent:otel-agent-1325.jar Testapp
NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[otel.javaagent 2024-11-14 11:32:43:733 +0000] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 1.32.5-aws
TestApp
[otel.javaagent 2024-11-14 11:32:48:798 +0000] [OkHttp http://localhost:4317/...] WARN io.opentelemetry.exporter.internal.grpc.GrpcExporter - Failed to export metrics. Server responded with gRPC status code 2. Error message: Failed to connect to localhost/127.0.0.1:4317

[bjrara]

Thanks for reporting the issue, we will look into it.

[bjrara]

Just to clarify, it's not a new problem introduced in the new version. I tested 1.32.4 and it didn't work with JDK23.

[nkvignesh]

I was using 1.28.1 with JDK17, no issues observed.
With JDK23, either 1.28.1 or the latest (1.32.5) didnt work.

Does that mean otel is not yet compatible with JDK23?

[bjrara]

ADOT SDK is currently based on 1.32.1 while the primary version of OTEL SDK is on 2.10.x.

I see the latest OTEL SDK is JDK23 compatible: https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/12329/files.

I believe it's an issue with 1.32.1. We are upgrading the ADOT SDK to 2.x and if it cannot catch up with the next release, we may want to fix the JDK 23 compatibility issue.

[nkvignesh]

what is the timeline for JDK23 compatibility issue fix for the ADOT libs?

[bjrara]

We will update here once we have a clear timeline.

[bjrara]

what is the timeline for JDK23 compatibility issue fix for the ADOT libs?

We plan to release a new version with JDK23 support in late December or early January.