Error: Port already in use. Version 1.43.0 or higher.

[ajax-boiko-y]

Description

After updating the agent version, the application occasionally fails to start due to a port conflict error. This issue appears to be a race condition where the application’s configured JMX remote port (8856) is temporarily used by jps during startup. This problem was not observed before the agent update.

Observed Behavior

• The application logs indicate a Port already in use error when attempting to bind to the JMX remote port.
• The issue seems transient—sometimes the port becomes available, and the application starts successfully.

Error Details

Error: JMX connector server communication error: service:jmx:rmi://stage-sp-tcp-ip-10-193-8-163:8856 jdk.internal.agent.AgentConfigurationError: java.rmi.server.ExportException: Port already in use: 8856; nested exception is: java.net.BindException: Address already in use at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:841) at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:479) at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:447) at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:603) Caused by: java.rmi.server.ExportException: Port already in use: 8856; nested exception is: java.net.BindException: Address already in use at java.rmi/sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:356) at java.rmi/sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:251) at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:415) at java.rmi/sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) at java.rmi/sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:235) at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap$PermanentExporter.exportObject(ConnectorBootstrap.java:204) at java.management.rmi/javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:153) at java.management.rmi/javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:138) at java.management.rmi/javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:450) at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:837) ... 3 more

Configuration Details

JAVA_TOOL_OPTIONS includes the following relevant options:
-Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.port=8855
-Dcom.sun.management.jmxremote.rmi.port=8856
-Dcom.sun.management.jmxremote.local.only=false
-Djava.rmi.server.hostname=localhost
-Djava.net.preferIPv4Stack=true

-javaagent:/usr/local/dd-java-agent/dd-java-agent.jar
-Ddd.env=stage-sp
-Ddd.service=tcp
-Ddd.version=1.1.0.12.RELEASE
-Ddd.trace.enabled=false
-Ddd.jmxfetch.enabled=false
-Ddd.profiling.enabled=true
-Ddd.profiling.ddprof.cpu.enabled=true
-Ddd.profiling.ddprof.wall.enabled=true
-Ddd.profiling.ddprof.alloc.enabled=true
-Ddd.profiling.ddprof.liveheap.enabled=true

Investigation Notes

• During startup, jps temporarily uses ports 8855 and 8856:

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
jps 31369 root 9u IPv4 102917 0t0 TCP *:8856 (LISTEN)
jps 31369 root 11u IPv4 102918 0t0 TCP *:8855 (LISTEN)

• Once jps releases the ports, the application starts normally.

Possible Cause

The problem might have been introduced by recent changes in the agent or its dependencies. A potential related change: dd-trace-java PR #7971.

Expected Behavior

The application should start consistently without encountering port conflicts.

Additional Information

• Does not occur with the previous agent version.
• The port conflict issue is intermittent and resolves if jps releases the ports in time.