Skip to content

Conversation

@3pacccccc
Copy link
Contributor

Main Issue: #24796

Motivation

When creating multiple PulsarAdmin clients in the same JVM, each client currently creates its own set of resources (event loops, timers, DNS resolvers). This leads to excessive thread consumption and memory usage when many admin clients are instantiated.

This change allows PulsarAdmin clients to share resources through the PulsarClientSharedResources mechanism, similar to what's already available for PulsarClient.

Modifications

  1. Added sharedResources(PulsarClientSharedResources) method to PulsarAdminBuilder API
  2. Modified PulsarAdminBuilderImpl to accept and propagate shared resources configuration
  3. Updated PulsarAdminImpl constructor to pass shared resources to HTTP connector
  4. Enhanced AsyncHttpConnector to utilize shared resources:
    • Event loop group for I/O operations
    • Netty timer for timeouts and scheduling
    • DNS resolver for name resolution
  5. Added test to verify shared resources are properly reused across multiple admin clients.

Verifying this change

  • Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: 3pacccccc#29

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Oct 25, 2025
@3pacccccc 3pacccccc closed this Oct 28, 2025
@3pacccccc 3pacccccc reopened this Oct 28, 2025
@3pacccccc 3pacccccc requested a review from lhotari October 28, 2025 08:52
@lhotari lhotari changed the title [feat][client]Support shared resources in PulsarAdmin to reduce thread usage [feat][client] PIP-234: Support shared resources in PulsarAdmin to reduce thread usage Oct 31, 2025
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Great work @3pacccccc

@lhotari lhotari requested a review from Copilot October 31, 2025 07:29
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for sharing resources (event loop groups, timers, and DNS resolvers) across multiple PulsarAdmin client instances to reduce memory footprint and thread usage when creating many clients in the same JVM.

Key changes:

  • Added sharedResources parameter to PulsarAdminBuilder and AsyncHttpConnector
  • Updated AsyncHttpConnector to manage shared and created event loop groups
  • Added new test cases to verify shared resource behavior

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
PulsarAdminBuilder.java Added API method to accept PulsarClientSharedResources
PulsarAdminBuilderImpl.java Implemented sharedResources method and stored shared resources field
PulsarAdminImpl.java Updated constructors to accept and pass shared resources to connector
AsyncHttpConnectorProvider.java Updated getConnector to accept shared resources parameter
AsyncHttpConnector.java Added logic to handle shared resources, including managing event loops, DNS resolvers, and cleanup
PulsarAdminBuilderImplTest.java Added test cases for shared resources functionality
AsyncHttpConnectorTest.java Updated test constructor calls to include null shared resources parameter

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@3pacccccc 3pacccccc requested a review from lhotari October 31, 2025 08:51
@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 84.78261% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.23%. Comparing base (676ba07) to head (9048d50).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
...client/admin/internal/http/AsyncHttpConnector.java 82.92% 2 Missing and 5 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##             master   #24893       +/-   ##
=============================================
+ Coverage     38.56%   74.23%   +35.67%     
- Complexity    13262    33877    +20615     
=============================================
  Files          1856     1913       +57     
  Lines        145287   149488     +4201     
  Branches      16877    17371      +494     
=============================================
+ Hits          56025   110973    +54948     
+ Misses        81696    29647    -52049     
- Partials       7566     8868     +1302     
Flag Coverage Δ
inttests 26.28% <43.47%> (+0.10%) ⬆️
systests 22.70% <43.47%> (-0.06%) ⬇️
unittests 73.75% <84.78%> (+39.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
.../client/admin/internal/PulsarAdminBuilderImpl.java 76.19% <100.00%> (+43.18%) ⬆️
.../pulsar/client/admin/internal/PulsarAdminImpl.java 79.04% <100.00%> (+10.47%) ⬆️
...dmin/internal/http/AsyncHttpConnectorProvider.java 88.88% <ø> (ø)
...client/admin/internal/http/AsyncHttpConnector.java 85.08% <82.92%> (+11.72%) ⬆️

... and 1407 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lhotari lhotari merged commit d186946 into apache:master Nov 4, 2025
53 checks passed
lhotari pushed a commit that referenced this pull request Nov 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants