locate: refactor RegionRequestSender.SendReqCtx by zyguan · Pull Request #1565 · tikv/client-go

zyguan · 2025-01-26T10:15:53Z

This PR extracts the iteration of retry loop in SendReqCtx into sendReqState.next. We will reuse sendReqState in async client API.

ref #1586

Signed-off-by: zyguan <zhongyangguan@gmail.com>

cfzjywxk

LGTM

Signed-off-by: zyguan <zhongyangguan@gmail.com>

ti-chi-bot · 2025-02-25T03:34:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfzjywxk, ekexium

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [cfzjywxk,ekexium]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-02-25T03:34:06Z

[LGTM Timeline notifier]

Timeline:

2025-02-18 03:48:02.106991621 +0000 UTC m=+933124.503213678: ☑️ agreed by cfzjywxk.
2025-02-25 03:34:05.535755435 +0000 UTC m=+326793.488913700: ☑️ agreed by ekexium.

you06 · 2025-02-28T01:45:46Z

This commit break the a test in TiDB.

Reproduce

tiup playground --tiflash=0 nightly --mode tikv-slim --kv=3

go test ./tests/realtikvtest/sessiontest -run TestTiKVClientReadTimeout --tags=intest --with-real-tikv=true

Log

    testkit.go:318: 
                Error Trace:    pkg/testkit/testkit.go:318
                                                        pkg/testkit/testkit.go:243
                                                        pkg/testkit/testkit.go:176
                                                        tests/realtikvtest/sessiontest/session_fail_test.go:287
                Error:          Received unexpected error:
                                [127.0.0.1:20160](1) wait recvLoop timeout, timeout:1ms: context deadline exceeded
                                github.com/pingcap/errors.AddStack
                                        external/com_github_pingcap_errors/errors.go:178
                                github.com/pingcap/errors.Trace
                                        external/com_github_pingcap_errors/juju_adaptor.go:15
                                github.com/pingcap/tidb/pkg/store/driver/error.ToTiDBErr
                                        pkg/store/driver/error/error.go:208
                                github.com/pingcap/tidb/pkg/store/copr.(*copIteratorWorker).handleTaskOnce
                                        pkg/store/copr/coprocessor.go:1426
                                github.com/pingcap/tidb/pkg/store/copr.(*liteCopIteratorWorker).liteSendReq
                                        pkg/store/copr/coprocessor.go:1197
                                github.com/pingcap/tidb/pkg/store/copr.(*copIterator).Next
                                        pkg/store/copr/coprocessor.go:1112
                                github.com/pingcap/tidb/pkg/distsql.(*selectResult).fetchResp
                                        pkg/distsql/select_result.go:317
                                github.com/pingcap/tidb/pkg/distsql.(*selectResult).Next
                                        pkg/distsql/select_result.go:382
                                github.com/pingcap/tidb/pkg/executor.(*tableResultHandler).nextChunk
                                        pkg/executor/table_reader.go:612
                                github.com/pingcap/tidb/pkg/executor.(*TableReaderExecutor).Next
                                        pkg/executor/table_reader.go:331
                                github.com/pingcap/tidb/pkg/executor/internal/exec.Next
                                        pkg/executor/internal/exec/executor.go:460
                                github.com/pingcap/tidb/pkg/executor.(*ExplainExec).executeAnalyzeExec
                                        pkg/executor/explain.go:129
                                github.com/pingcap/tidb/pkg/executor.(*ExplainExec).generateExplainInfo
                                        pkg/executor/explain.go:148
                                github.com/pingcap/tidb/pkg/executor.(*ExplainExec).Next
                                        pkg/executor/explain.go:75
                                github.com/pingcap/tidb/pkg/executor/internal/exec.Next
                                        pkg/executor/internal/exec/executor.go:460
                                github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next
                                        pkg/executor/adapter.go:1269
                                github.com/pingcap/tidb/pkg/executor.(*recordSet).Next
                                        pkg/executor/adapter.go:172
                                github.com/pingcap/tidb/pkg/session.GetRows4Test
                                        pkg/session/tidb.go:343
                                github.com/pingcap/tidb/pkg/session.ResultSetToStringSlice
                                        pkg/session/tidb.go:361
                                github.com/pingcap/tidb/pkg/testkit.(*TestKit).ResultSetToResultWithCtx
                                        pkg/testkit/testkit.go:317
                                github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQueryWithContext
                                        pkg/testkit/testkit.go:243
                                github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQuery
                                        pkg/testkit/testkit.go:176
                                tests/realtikvtest/sessiontest/sessiontest_test.TestTiKVClientReadTimeout
                                        tests/realtikvtest/sessiontest/session_fail_test.go:287
                                testing.tRunner
                                        GOROOT/src/testing/testing.go:1690
                                runtime.goexit
                                        src/runtime/asm_amd64.s:1700
                Test:           TestTiKVClientReadTimeout
                Messages:       sql:explain analyze select /*+ set_var(tikv_client_read_timeout=1) */ * from t where b > 1, args:[]

* *: bump pd client (tikv#1575) Signed-off-by: Ryan Leung <rleungx@gmail.com> * OWNERS: Auto Sync OWNERS files from community membership (tikv#1576) Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> * sync etcd endpoints immediately after initializing the client (tikv#1573) Signed-off-by: Vlad Dmitriev <vldmit@gmail.com> * p-dml: resolve locks concurrently (tikv#1584) close tikv#1577 Signed-off-by: you06 <you1474600@gmail.com> * memdb: prevent iterator invalidation (tikv#1563) ref pingcap/tidb#59153 Signed-off-by: ekexium <eke@fastmail.com> * locate: refactor RegionRequestSender.SendReqCtx (tikv#1565) Signed-off-by: zyguan <zhongyangguan@gmail.com> * locate: fix the default settings of circuit breaker (tikv#1593) ref tikv/pd#8678 Signed-off-by: Ryan Leung <rleungx@gmail.com> * util: define and implement core interfaces for async api (tikv#1591) ref tikv#1586 Signed-off-by: zyguan <zhongyangguan@gmail.com> * Add a retry when getting ts from PD for validating read ts (tikv#1600) Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * pdclient: Add caller info to pd client (tikv#1516) ref tikv/pd#8593 Signed-off-by: okJiang <819421878@qq.com> * locate: fix TestTiKVClientReadTimeout (tikv#1601) Signed-off-by: zyguan <zhongyangguan@gmail.com> * *: update pd client (tikv#1605) Signed-off-by: Ryan Leung <rleungx@gmail.com> * Validate ts only for stale read (tikv#1607) ref pingcap/tidb#59402 Signed-off-by: ekexium <eke@fastmail.com> * execdetails: export scheduler write details (tikv#1606) Signed-off-by: Neil Shen <overvenus@gmail.com> * Update pd client (tikv#1615) Signed-off-by: disksing <i@disksing.com> * ci: allow use label to skip integration tests (tikv#1616) Signed-off-by: disksing <i@disksing.com> * remove useless metric tidb_tikvclient_cop_duration_seconds_bucket (tikv#1602) Signed-off-by: XuHuaiyu <391585975@qq.com> * client: implement SendRequestAsync for RPCClient (tikv#1604) ref tikv#1586 Signed-off-by: zyguan <zhongyangguan@gmail.com> * execdetails: export grpc process and wait time to time details (tikv#1614) Signed-off-by: Neil Shen <overvenus@gmail.com> Co-authored-by: Bisheng Huang <hbisheng@gmail.com> * Refine pessimistic lock related metrics and stats (tikv#1620) Signed-off-by: yibin87 <huyibin@pingcap.com> * metrics: adjust bucket count to reduce metrics data (tikv#1609) Signed-off-by: Lynn <zimu_xia@126.com> * update tidb for integration tests (tikv#1621) Signed-off-by: disksing <i@disksing.com> * support redact key in logs (tikv#1612) ref pingcap/tidb#59279 Signed-off-by: tangenta <tangenta@126.com> Co-authored-by: you06 <you1474600@gmail.com> * update integration_test/go.mod (tikv#1624) Signed-off-by: tangenta <tangenta@126.com> * memdb: introduce snapshot interface (tikv#1623) Signed-off-by: you06 <you1474600@gmail.com> Co-authored-by: ekexium <eke@fastmail.com> * pd: enable OutputMustContainAllKeyRange (tikv#1632) Signed-off-by: lhy1024 <admin@liudos.us> * Fix backoff lose info when forked (tikv#1627) ref pingcap/tidb#60271 Signed-off-by: yibin87 <huyibin@pingcap.com> * tikv: disable health-feedback in next-gen (tikv#1635) Signed-off-by: zyguan <zhongyangguan@gmail.com> * enable ts validation for normal read (tikv#1619) Signed-off-by: ekexium <eke@fastmail.com> * Add txn write conflict metrics (tikv#1551) close tikv#1550 Signed-off-by: sujuntao <juntao.su@foxmail.com> Co-authored-by: sujuntao <juntao.su@foxmail.com> * apicodec: fix a typo when encoding request for CmdMvccGetByKey (tikv#1638) Signed-off-by: tiancaiamao <tiancaiamao@gmail.com> Co-authored-by: cfzjywxk <lsswxrxr@163.com> * *: update kvproto version (tikv#1636) ref tikv#1631 Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * txn: provide more information in commit RPC / log mvcc debug info when commit failed for `TxnLockNotFound` (tikv#1640) ref tikv#1631 Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * txn: handle undetermined error in client go (tikv#1642) close tikv#1641 Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * txn: fix the implemention of undetermined error (tikv#1644) close tikv#1641 Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * locate: implement SendReqAsync for RegionRequestSender (tikv#1618) ref tikv#1586 Signed-off-by: zyguan <zhongyangguan@gmail.com> * update pd client for resource group and keyspace (tikv#1645) Signed-off-by: lhy1024 <admin@liudos.us> * tests: bump tidb to fix integration tests (tikv#1650) Signed-off-by: zyguan <zhongyangguan@gmail.com> * Fix some metrics that miss const labels (tikv#1652) Signed-off-by: yibin87 <huyibin@pingcap.com> * Replace etcd safe point with txn safe point for read safety check (tikv#1634) Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Fix stale read metrics (tikv#1649) close tikv#1648 Signed-off-by: you06 <you1474600@gmail.com> * *: support async batch get (tikv#1646) ref tikv#1586 Signed-off-by: zyguan <zhongyangguan@gmail.com> * Update kvproto dependancy and set keyspace name for rpc context (tikv#1667) close tikv#1668 Signed-off-by: yibin87 <huyibin@pingcap.com> * ci: add next-gen integration tests (tikv#1661) Signed-off-by: ekexium <eke@fastmail.com> * snapshot: set `ReplicaRead` to false when `ReplicaReadType` fallbacks to `ReplicaReadLeader` (tikv#1663) ref pingcap/tidb#61745 Signed-off-by: you06 <you1474600@gmail.com> * resource_control: support collecting cross AZ traffic in ru consumption (tikv#1669) Signed-off-by: glorv <glorvs@163.com> * txnkv: prevent some actions from being interrupted by kill (tikv#1665) fix pingcap/tidb#61454 Signed-off-by: zyguan <zhongyangguan@gmail.com> * region_cache: add ForceRefreshAllStores function (tikv#1686) Signed-off-by: guo-shaoge <shaoge1994@163.com> * upgrade gRPC to allow consumption by peerdb * Bump gprc version to 1.73.0 Remove references to `grpc.NewSharedBufferPool()`, it was removed from the grpc package and causes build to fail - it's always on. (https://pkg.go.dev/google.golang.org/grpc/experimental#WithBufferPool) Signed-off-by: Tiago Scolari <git@tscolari.me> * go mod tidy --------- Signed-off-by: Ryan Leung <rleungx@gmail.com> Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Signed-off-by: Vlad Dmitriev <vldmit@gmail.com> Signed-off-by: you06 <you1474600@gmail.com> Signed-off-by: ekexium <eke@fastmail.com> Signed-off-by: zyguan <zhongyangguan@gmail.com> Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Signed-off-by: okJiang <819421878@qq.com> Signed-off-by: Neil Shen <overvenus@gmail.com> Signed-off-by: disksing <i@disksing.com> Signed-off-by: XuHuaiyu <391585975@qq.com> Signed-off-by: yibin87 <huyibin@pingcap.com> Signed-off-by: Lynn <zimu_xia@126.com> Signed-off-by: tangenta <tangenta@126.com> Signed-off-by: lhy1024 <admin@liudos.us> Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> Signed-off-by: glorv <glorvs@163.com> Signed-off-by: guo-shaoge <shaoge1994@163.com> Signed-off-by: Tiago Scolari <git@tscolari.me> Co-authored-by: Ryan Leung <rleungx@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Co-authored-by: Vlad Dmitriev <vldmit@gmail.com> Co-authored-by: you06 <you1474600@gmail.com> Co-authored-by: ekexium <eke@fastmail.com> Co-authored-by: zyguan <zhongyangguan@gmail.com> Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com> Co-authored-by: okJiang <819421878@qq.com> Co-authored-by: Neil Shen <overvenus@gmail.com> Co-authored-by: disksing <i@disksing.com> Co-authored-by: HuaiyuXu <xuhuaiyu@pingcap.com> Co-authored-by: Bisheng Huang <hbisheng@gmail.com> Co-authored-by: yibin <huyibin@pingcap.com> Co-authored-by: Lynn <zimu_xia@126.com> Co-authored-by: tangenta <tangenta@126.com> Co-authored-by: lhy1024 <lhylhy1024@gmail.com> Co-authored-by: JT <43174723+ImSjt@users.noreply.github.com> Co-authored-by: sujuntao <juntao.su@foxmail.com> Co-authored-by: tiancaiamao <tiancaiamao@gmail.com> Co-authored-by: cfzjywxk <lsswxrxr@163.com> Co-authored-by: 王超 <cclcwangchao@hotmail.com> Co-authored-by: glorv <glorvs@163.com> Co-authored-by: guo-shaoge <shaoge1994@163.com> Co-authored-by: Kevin Biju <kevin@peerdb.io> Co-authored-by: Tiago Scolari <tiago@diagrid.io>

Signed-off-by: zyguan <zhongyangguan@gmail.com>

locate: refactor RegionRequestSender.SendReqCtx

9f08a5c

Signed-off-by: zyguan <zhongyangguan@gmail.com>

ti-chi-bot Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the dco. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 26, 2025

zyguan added 3 commits January 26, 2025 10:27

try to fix ut

b571c08

Signed-off-by: zyguan <zhongyangguan@gmail.com>

fix an anothor issue

d6ce406

Signed-off-by: zyguan <zhongyangguan@gmail.com>

fix TestIsRetryRequestFlagWithRegionError

0b6c751

Signed-off-by: zyguan <zhongyangguan@gmail.com>

zyguan marked this pull request as ready for review January 26, 2025 13:41

ti-chi-bot Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 26, 2025

cfzjywxk requested review from cfzjywxk, crazycs520, ekexium and you06 February 17, 2025 07:04

cfzjywxk approved these changes Feb 18, 2025

View reviewed changes

ti-chi-bot Bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Feb 18, 2025

ekexium reviewed Feb 18, 2025

View reviewed changes

Comment thread internal/locate/region_request.go

add some comments to sendReqState

7498d18

Signed-off-by: zyguan <zhongyangguan@gmail.com>

cfzjywxk requested a review from ekexium February 24, 2025 07:29

ekexium approved these changes Feb 25, 2025

View reviewed changes

ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Feb 25, 2025

ti-chi-bot Bot merged commit 8f40417 into tikv:master Feb 25, 2025

zyguan mentioned this pull request Feb 21, 2025

Support async send requests #1586

Open

zyguan mentioned this pull request Feb 28, 2025

locate: fix TestTiKVClientReadTimeout #1601

Merged

zyguan added a commit to ti-chi-bot/client-go that referenced this pull request Dec 2, 2025

locate: refactor RegionRequestSender.SendReqCtx (tikv#1565)

b14ad6d

Signed-off-by: zyguan <zhongyangguan@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

locate: refactor RegionRequestSender.SendReqCtx#1565

locate: refactor RegionRequestSender.SendReqCtx#1565
ti-chi-bot[bot] merged 5 commits intotikv:masterfrom
zyguan:dev/refactor-send-req

zyguan commented Jan 26, 2025 •

edited

Loading

Uh oh!

cfzjywxk left a comment

Uh oh!

Uh oh!

ti-chi-bot Bot commented Feb 25, 2025

Uh oh!

ti-chi-bot Bot commented Feb 25, 2025

Uh oh!

you06 commented Feb 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

zyguan commented Jan 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cfzjywxk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ti-chi-bot Bot commented Feb 25, 2025

Uh oh!

ti-chi-bot Bot commented Feb 25, 2025

[LGTM Timeline notifier]

Uh oh!

you06 commented Feb 28, 2025

Reproduce

Log

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zyguan commented Jan 26, 2025 •

edited

Loading