Skip to content

Disconnect the redis client the metrics consumer opens - #2850

Merged
bert-e merged 1 commit into
development/9.6from
bugfix/BB-882-metricsconsumer-redis-clea
Sep 15, 2026
Merged

bert-e merged 1 commit into
development/9.6from
bugfix/BB-882-metricsconsumer-redis-clea

Conversation

@francoisferrand

Copy link
Copy Markdown
Contributor

MetricsConsumer built its RedisClient as a constructor local and handed it straight to the StatsModel, which keeps it as its private _redis. Nothing in backbeat could reach it afterwards, and close() only tore down the kafka consumer — so the socket stayed open until the process died. One lingering connection per populator process.

Keeping the client as a member is enough to fix it: close() now disconnects it alongside the consumer, on both the success and error paths. Both owners already call _mConsumer.close() from their own close paths (QueuePopulator, IngestionPopulator), so there was nothing to wire up on their side.

While in there: close() dereferenced this._consumer unconditionally, but start() only assigns it on the consumer's ready event. Closing during startup threw a TypeError and left the caller's async.series waiting on a callback that never came. Guarded, and it still calls back.

Adds a unit test for MetricsConsumer, which had none.

Scope note: this is the same class of leak as BB-320, but a distinct component and an independent branch — no overlap with the files that PR touches. It does not make mocha --exit removable on its own; that needs BB-320 and at least the getRedisClient.js leaks too, tracked as BB-321.

Issue: BB-882

@bert-e

bert-e commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Hello francoisferrand,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/bypass_source_branch_lineage Bypass the cross-branch contamination check
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request.
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.99%. Comparing base (6778296) to head (f1e21b8).

Additional details and impacted files

Impacted file tree graph

Files with missing lines Coverage Δ
lib/MetricsConsumer.js 32.65% <100.00%> (+4.69%) ⬆️

... and 3 files with indirect coverage changes

Components Coverage Δ
Bucket Notification 80.25% <ø> (ø)
Core Library 83.31% <100.00%> (+0.52%) ⬆️
Ingestion 75.65% <ø> (ø)
Lifecycle 81.25% <ø> (ø)
Oplog Populator 85.80% <ø> (ø)
Replication 63.75% <ø> (ø)
Bucket Scanner 85.76% <ø> (ø)
@@                 Coverage Diff                 @@
##           development/9.6    #2850      +/-   ##
===================================================
+ Coverage            76.77%   76.99%   +0.21%     
===================================================
  Files                  205      205              
  Lines                14288    14293       +5     
===================================================
+ Hits                 10970    11005      +35     
+ Misses                3308     3278      -30     
  Partials                10       10              
Flag Coverage Δ
api:retry 9.28% <0.00%> (-0.01%) ⬇️
api:routes 9.06% <0.00%> (-0.01%) ⬇️
bucket-scanner 85.76% <ø> (ø)
ft_test:queuepopulator 11.01% <25.00%> (+1.22%) ⬆️
ingestion 12.86% <0.00%> (-0.01%) ⬇️
lib 9.03% <0.00%> (-0.03%) ⬇️
lifecycle 19.33% <0.00%> (-0.03%) ⬇️
notification 0.99% <0.00%> (-0.01%) ⬇️
oplogPopulator 0.13% <0.00%> (-0.01%) ⬇️
replication 19.05% <0.00%> (-0.01%) ⬇️
unit 58.15% <100.00%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread lib/MetricsConsumer.js
@scality scality deleted a comment from bert-e Sep 14, 2026
@bert-e

bert-e commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@francoisferrand
francoisferrand requested review from a team, benzekrimaha and delthas September 14, 2026 09:32
MetricsConsumer built its RedisClient as a local and handed it straight to
the StatsModel, so nothing could reach it afterwards: close() only tore down
the kafka consumer, and the socket stayed open for the life of the process.
Keep the client as a member and disconnect it alongside the consumer. Both
populators already close their metrics consumer, so there is nothing to wire
up on their side.

close() also dereferenced the consumer unconditionally, while start() only
assigns it on the ready event: closing during startup threw a TypeError and
left the caller's series waiting on a callback that never came. Guard it, and
still call back.

Issue: BB-882
@francoisferrand
francoisferrand changed the base branch from development/9.6 to improvement/BB-804 September 15, 2026 16:36
@bert-e
bert-e deleted the branch development/9.6 September 15, 2026 16:36
@bert-e bert-e closed this Sep 15, 2026
@francoisferrand
francoisferrand changed the base branch from improvement/BB-804 to development/9.6 September 15, 2026 17:18
@francoisferrand
francoisferrand force-pushed the bugfix/BB-882-metricsconsumer-redis-clea branch from 4ac8887 to f1e21b8 Compare September 15, 2026 17:19
@francoisferrand

Copy link
Copy Markdown
Contributor Author

/approve

@bert-e

bert-e commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/9.6

The following branches have NOT changed:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.6
  • development/9.0
  • development/9.1
  • development/9.2
  • development/9.3
  • development/9.4
  • development/9.5

This pull request did not target the following hotfix branch(es) so they
were left untouched:

  • hotfix/7.10.8
  • hotfix/9.0.7
  • hotfix/7.4.4
  • hotfix/7.4.5
  • hotfix/7.4.6
  • hotfix/7.8.0
  • hotfix/7.9.0
  • hotfix/7.4.2
  • hotfix/7.10.4
  • hotfix/7.4.7
  • hotfix/7.6.0
  • hotfix/7.4.8
  • hotfix/7.4.10
  • hotfix/7.4.9
  • hotfix/7.4.3
  • hotfix/7.2.0
  • hotfix/7.10.0
  • hotfix/7.10.12
  • hotfix/7.70.12
  • hotfix/7.7.0
  • hotfix/7.10.1
  • hotfix/9.0.4
  • hotfix/8.2.12
  • hotfix/7.70.15
  • hotfix/7.4.0
  • hotfix/7.4.1
  • hotfix/7.10.2
  • hotfix/7.10.3
  • hotfix/7.70.1
  • hotfix/7.10.17

Please check the status of the associated issue BB-882.

Goodbye francoisferrand.

The following options are set: approve

@bert-e
bert-e merged commit f1e21b8 into development/9.6 Sep 15, 2026
42 checks passed
@bert-e
bert-e deleted the bugfix/BB-882-metricsconsumer-redis-clea branch September 15, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants