Skip to content

Conversation

@zimulala
Copy link
Contributor

@zimulala zimulala commented Nov 26, 2025

What problem does this PR solve?

This is a small part extracted from PR #123.

What is changed and how it works?

  • If pulling top-heavy metrics fails, you can consider reducing the time range for each request. The metrics-low-priority flag is used to reduce the time range of each request.

    • However, this may result in much slower data retrieval.
  • Optimized connection pool management (use default http.Client)

    • By customizing the connection pool size and layered timeouts, blocked connections are prevented from affecting requests, improving concurrency stability.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    tiup diag collect debug \
  --exclude "log,system,db_vars" \
  --include "monitor" \
  --output "/root/prom-2k" \
  --from "2025-11-25T16:50:14+08:00" \
  --to "2025-11-25T17:10:14+08:00" \
  -R "tidb"

Before this PR

In this case, one file every 5 minutes, and the following three tests only get 5 minutes of data for each
截屏2025-11-26 10 15 07

After this PR

with --metrics-min-interval "300"
In this case, one file every 5 minutes, and the following three tests get 10 minutes of data for each
截屏2025-11-26 14 42 04

with default metrics-min-interval(120)
In this case, one file every 2 minutes, and we have 3 runs, the first with 12 minutes of data (6 files), and the next two tests with 20 minutes of data (10 files)
截屏2025-11-26 10 15 37

  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Copy link

@yibin87 yibin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@kaaaaaaang kaaaaaaang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGT @mornyx

subdirRaw = "raw"
maxQueryRange = 120 * 60 // 120min
minQueryRange = 5 * 60 // 5min
minQueryRange = 1 * 60 // 1min
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is not used anymore

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the minimum protection value. Because if the value is lower than this, the proportion of metadata will be high. If the segmentation is too small, it would be meaningless.

cmd.Flags().IntVar(&cOpt.MetricsLimit, "metricslimit", 10000, "metric size limit of single request, specified in series*hour per request")
cmd.Flags().StringVar(&metricsConf, "metricsconfig", "", "config file of metricsfilter")
cmd.Flags().StringSliceVar(&labels, "metricslabel", nil, "only collect metrics that match labels")
cmd.Flags().IntVar(&cOpt.MetricsMinInterval, "metrics-min-interval", 120, "the minimum interval of a single request in seconds")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is 120s the new default value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if you think it's best to keep it the same as before, I can change it to 5 minutes.

@zimulala
Copy link
Contributor Author

zimulala commented Dec 4, 2025

PTAL @mornyx

@kaaaaaaang kaaaaaaang merged commit f3f4770 into pingcap:master Dec 4, 2025
3 checks passed
@zimulala zimulala deleted the zimuxia/min-interval branch December 4, 2025 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants