Introduce RetryLimiter

[jrhee17]

Motivation:

This changeset attempts to solve the same problem as #6318.

Retry limiting is a concept which limits the number of retries in case a system undergoes a prolonged period of service degradation.
gRPC offers a token-based configuration which limits retries depending on certain predicates, whereas envoy offers a simple concurrency limiting configuration based on the number of active retries.

To support token-based retry limiters, I propose that RetryDecision#permits is added as metadata which could signal to RetryLimiter whether retries should be further made. This could be useful for systems behind load balancers, as the load balancer may return certain status codes depending on the health upstream.

To support simple concurrency-based retry limiters, I propose that a RetryLimiter#shouldRetry method is called right before a retry is executed.

Modifications:

• Introduced RetryLimiter which acts as an extension to dynamically limit retries.
  • RetryLimiter#shouldRetry decides whether a retry should be executed
  • RetryLimiter#handleDecision is invoked when a RetryDecision is made. RetryDecision#permits may be used to update the internal state and decide whether retries should be allowed.
• Added RetryDecision#permits
• Added APIs so users can set RetryLimiter to RetryConfig

Result:

• Users can specify RetryLimiter to limit retry requests.
• Closes Enable retry throttling #6282

[github-actions]

🔍 Build Scan® (commit: 35f7573)

Job name	Status	Build Scan®
build-ubicloud-standard-16-jdk-8	✅	https://ge.armeria.dev/s/7qolgyzbh5zia
build-ubicloud-standard-16-jdk-21-snapshot-blockhound	✅	https://ge.armeria.dev/s/i7jte2attycqq
build-ubicloud-standard-16-jdk-17-min-java-17-coverage	✅	https://ge.armeria.dev/s/idus7tdopndwi
build-ubicloud-standard-16-jdk-17-min-java-11	✅	https://ge.armeria.dev/s/grmoiok7r3vfc
build-ubicloud-standard-16-jdk-17-leak	✅	https://ge.armeria.dev/s/lb6cwrqhvmiqo
build-ubicloud-standard-16-jdk-11	✅	https://ge.armeria.dev/s/cmtepy3hyqfru
build-macos-latest-jdk-21	❌ (failure)	https://ge.armeria.dev/s/aq5gjxtjd5rue

[ikhoon]

Overall looks good.

[ikhoon]

Optional) Would it be useful to support rateLimiting by default?

[jrhee17]

I forgot to leave a comment on this - the RetryLimiter API doesn't go well with guava's RateLimiter as there is no way to determine if a the state should be rate-limited at a given time.

I'd like to revisit this later on unless I'm missing something/you have a better idea.

[ikhoon]

It seems that int is more appropriate type for maxTokens. Is there a reason a float is being used?
https://github.com/grpc/proposal/blob/master/A6-client-retries.md#validation-of-retrythrottling

maxTokens MUST be specified and MUST be a JSON integer value in the range (0, 1000].

[minwoox]

The Javadoc says

each positive {@link RetryDecision#permits()} will increment available tokens by {@code tokenRatio} 

Which is different from the actual logic.
Does the permit always -1, 0, 1?
I'm asking because this logic doesn't take into account the permit numbers, so I was wondering if we could use more specific names than 'permit'.

[jrhee17]

I've generalized tokenBased. Although users who prefer gRPC-style retry limiting will need to add a little more logic, the probably makes more sense for general users.

[codecov]

Codecov Report

❌ Patch coverage is 75.22124% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.13%. Comparing base (8150425) to head (35f7573).
⚠️ Report is 216 commits behind head on main.

Files with missing lines	Patch %	Lines
...m/linecorp/armeria/client/retry/RetryDecision.java	50.00%	10 Missing ⚠️
...p/armeria/client/retry/TokenBasedRetryLimiter.java	65.21%	5 Missing and 3 partials ⚠️
...ria/client/retry/ConcurrencyBasedRetryLimiter.java	66.66%	4 Missing and 1 partial ⚠️
...ecorp/armeria/client/retry/RetryConfigBuilder.java	66.66%	2 Missing ⚠️
...rp/armeria/client/retry/RetryLimitedException.java	80.00%	0 Missing and 1 partial ⚠️
...m/linecorp/armeria/client/retry/RetryLimiters.java	94.11%	1 Missing ⚠️
...necorp/armeria/client/retry/RetryingRpcClient.java	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6409      +/-   ##
============================================
- Coverage     74.46%   74.13%   -0.33%     
- Complexity    22234    23036     +802     
============================================
  Files          1963     2067     +104     
  Lines         82437    86222    +3785     
  Branches      10764    11323     +559     
============================================
+ Hits          61385    63924    +2539     
- Misses        15918    16881     +963     
- Partials       5134     5417     +283     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ikhoon]

Thanks! 👍 👍

[minwoox]

👍 👍 👍