Skip to content

Commit a421668

Browse files
committed
fix: improve driver-side timeout logging with timeout value and pool state
When OperationTimedOutException fires in SpeculativeExecution.onTimeout(), log the configured driver-side timeout, per-connection in-flight count, pool pending borrow queue length, and total pool in-flight count. This helps distinguish between two failure modes: - Pool contention (requests queued inside the driver before reaching server): visible as pool pending borrows > 0 - Server-side slowness (requests reached server but driver gave up first): visible as pool pending borrows = 0 with high connection in-flight To expose the timeout value, ResponseHandler.readTimeoutMillis is widened from private to package-private. All access remains within com.datastax.driver.core. Motivated by CUSTOMER-301 (Whoop): client-side OperationTimedOutException bursts with no corresponding server-side timeouts, requiring diagnostics to determine whether the bottleneck is in the driver pool or on the server.
1 parent fa0150c commit a421668

2 files changed

Lines changed: 15 additions & 1 deletion

File tree

driver-core/src/main/java/com/datastax/driver/core/Connection.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1887,7 +1887,7 @@ static class ResponseHandler {
18871887
final int streamId;
18881888
final ResponseCallback callback;
18891889
final int retryCount;
1890-
private final long readTimeoutMillis;
1890+
final long readTimeoutMillis;
18911891

18921892
private final long startTime;
18931893
private volatile Timeout timeout;

driver-core/src/main/java/com/datastax/driver/core/RequestHandler.java

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1007,6 +1007,20 @@ public boolean onTimeout(Connection connection, long latency, int retryCount) {
10071007

10081008
Host queriedHost = current;
10091009

1010+
HostConnectionPool pool = manager.pools.get(queriedHost);
1011+
logger.warn(
1012+
"[{}] Driver-side timeout waiting for response from {} — "
1013+
+ "configured timeout: {}ms, "
1014+
+ "connection in-flight: {}, "
1015+
+ "pool pending borrows: {}, "
1016+
+ "pool total in-flight: {}",
1017+
id,
1018+
connection.endPoint,
1019+
connectionHandler != null ? connectionHandler.readTimeoutMillis : "unknown",
1020+
connection.inFlight.get(),
1021+
pool != null ? pool.pendingBorrowCount.get() : "unknown",
1022+
pool != null ? pool.totalInFlight.get() : "unknown");
1023+
10101024
OperationTimedOutException timeoutException =
10111025
new OperationTimedOutException(
10121026
connection.endPoint, "Timed out waiting for server response");

0 commit comments

Comments
 (0)