Skip to content

mysqlctl: add CloneFromDonor #19064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
nickvanw merged 32 commits into from
Jan 13, 2026
Merged

mysqlctl: add CloneFromDonor #19064

nickvanw merged 32 commits into from
Jan 13, 2026
+456 −0

Conversation

@maxenglander
Copy link
Collaborator

@maxenglander maxenglander commented Dec 20, 2025 •
edited
Loading

Description

Next step after #19062.

That PR is adapted from @nickvanw's work in a private fork. Adds the base infrastructure for using CLONE at the MysqlDaemon level. This PR carries that work one level up, to the topology.

CloneFromDonor reads CLI flags --clone-from-primary and --clone-from-tablet. Depending on which of the two options is chosen, it uses topology to idenfity the host and port of the selected donor, and then uses CloneExecutor to perform the clone from that donor to the local recipient.

Related Issue(s)

Checklist

  • "Backport to:" labels have been added if this change should be back-ported to release branches
  • If this change is to be back-ported to previous releases, a justification is included in the PR description
  • Tests were added or are not required
  • Did the new or modified tests pass consistently locally and on CI?
  • Documentation was added or is not required

Deployment Notes

AI Disclosure

AI was used.

nickvanw and others added 6 commits December 19, 2025 15:23
@nickvanw
mysqlctl: add MySQL CLONE support infrastructure
5458bcd 
Add core infrastructure for MySQL CLONE REMOTE operations:

- CloneExecutor in mysqlctl/clone.go for executing CLONE INSTANCE FROM
- Clone user configuration in dbconfigs (--db-clone-user, --db-clone-password, --db-clone-use-ssl)
- vt_clone user in init_db.sql with BACKUP_ADMIN privilege (MySQL 8.0.17+ only)
- Clone plugin loading in mycnf configs (plugin-load-add = mysql_clone.so)
- MySQLClonePluginFlavorCapability for version checking

MySQL CLONE copies data at the physical level over the network, providing
a faster alternative to logical backup/restore for replica provisioning.
Requires MySQL 8.0.17+ and InnoDB-only tables.

Signed-off-by: Nick Van Wiggeren <[email protected]>
@nickvanw
add flags to tests + fix errors from linters
0fe8efd 
Signed-off-by: Nick Van Wiggeren <[email protected]>
@nickvanw
fix up more tests
8941459 
Signed-off-by: Nick Van Wiggeren <[email protected]>
@maxenglander
go/vt/mysqlctl: add CloneFromDonor
94624dc 
Signed-off-by: Max Englander <[email protected]>
@nickvanw
actually validate the source and destination, and poll for the copy t…
0342d74 
…o be marked as completed after the CLONE is done

Signed-off-by: Nick Van Wiggeren <[email protected]>
@maxenglander
TestCloneFromDonor: add a few cases
95dccd1 
Signed-off-by: Max Englander <[email protected]>
@github-actions github-actions bot added this to the v24.0.0 milestone Dec 20, 2025
@vitess-bot vitess-bot bot added NeedsWebsiteDocsUpdate What it says NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsBackportReason If backport labels have been applied to a PR, a justification is required labels Dec 20, 2025
@vitess-bot
Copy link
Contributor

vitess-bot bot commented Dec 20, 2025

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

  • Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

  • Apply the release notes (needs details) label if users need to know about this change.
  • New features should be documented.
  • There should be some code comments as to why things are implemented the way they are.
  • There should be a comment at the top of each new or modified test to explain what the test does.

New flags

  • Is this flag really necessary?
  • Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
  • vtctl command output order should be stable and awk-able.

@maxenglander
merge <- clone-infrastructure
ba66344 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
more tests
e66d1ee 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
make it nicer
0d57e26 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
make tests nicer
06a370b 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
slight reorg add comments
88340e5 
Signed-off-by: Max Englander <[email protected]>
@maxenglander maxenglander added Type: Feature Component: Backup and Restore and removed NeedsIssue A linked issue is missing for this Pull Request NeedsBackportReason If backport labels have been applied to a PR, a justification is required labels Dec 20, 2025
@maxenglander
fix imports
01fabc0 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
fix lint
ca83a7a 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
one more lint
2cdb9c4 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
tidy
66e4998 
Signed-off-by: Max Englander <[email protected]>
@codecov
Copy link

codecov bot commented Dec 23, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 88.23529% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.90%. Comparing base (32b8bd8) to head (e63dc67).
⚠️ Report is 19 commits behind head on main.

Files with missing lines Patch % Lines
go/vt/mysqlctl/clone.go 87.75% 6 Missing ⚠️
Additional details and impacted files 
@@           Coverage Diff            @@
##             main   #19064    +/-   ##
========================================
  Coverage   69.89%   69.90%            
========================================
  Files        1612     1613     +1     
  Lines      215826   216076   +250     
========================================
+ Hits       150857   151044   +187     
- Misses      64969    65032    +63

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@maxenglander maxenglander marked this pull request as ready for review December 23, 2025 18:57
@maxenglander maxenglander mentioned this pull request Dec 24, 2025
Open
5 tasks
Base automatically changed from clone-infrastructure to main January 9, 2026 16:46
@nickvanw nickvanw requested a review from shlomi-noach as a code owner January 9, 2026 16:46
@maxenglander
merge <- main
2be837f 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
fix imports
1634726 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
tidy
6021d76 
Signed-off-by: Max Englander <[email protected]>
nickvanw
nickvanw reviewed Jan 9, 2026
View reviewed changes
nickvanw
nickvanw reviewed Jan 9, 2026
View reviewed changes
maxenglander and others added 3 commits January 9, 2026 14:46
@maxenglander @nickvanw
Update go/vt/mysqlctl/clone.go
cbd004b 
Co-authored-by: Nick Van Wiggeren <[email protected]>
Signed-off-by: Max Englander <[email protected]>
@maxenglander
configurable clone restart timeout
b8c0027 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
update tests
bf13376 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
one more test case
c46e365 
Signed-off-by: Max Englander <[email protected]>
@nickvanw nickvanw removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says labels Jan 10, 2026
@maxenglander maxenglander requested a review from nickvanw January 10, 2026 01:38
nickvanw
nickvanw approved these changes Jan 10, 2026
View reviewed changes
Copy link
Collaborator

@nickvanw nickvanw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, I suggest we write website documentation and changelog entries when the stacked PRs are fully complete.

mattlord
mattlord approved these changes Jan 12, 2026
View reviewed changes
Copy link
Member

@mattlord mattlord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! ❤️ Just a couple of minor comments.

go/vt/mysqlctl/clone.go Outdated
return replication.Position{}, errors.New("--clone-from-primary and --clone-from-tablet are mutually exclusive")
case cloneFromPrimary:
// Look up the primary tablet from topology.
log.Infof("Looking up primary tablet for shard %s/%s", keyspace, shard)
Copy link
Member

@mattlord mattlord Jan 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth adding some context like "for use as CLONE REMOTE target" to the log messages. For the errors returned, we can/should wrap them with that context at the CloneFromDonor call site (there isn't one in this PR so we have to maintain that context ourselves across the PRs).

maxenglander and others added 2 commits January 12, 2026 15:21
@maxenglander @mattlord
Update go/vt/mysqlctl/clone.go
ceef59d 
Co-authored-by: Matt Lord <[email protected]>
Signed-off-by: Max Englander <[email protected]>
@maxenglander
improve logging
2089cf2 
Signed-off-by: Max Englander <[email protected]>
@maxenglander
trigger ci
e63dc67 
Signed-off-by: Max Englander <[email protected]>
@nickvanw nickvanw enabled auto-merge (squash) January 13, 2026 21:55
@nickvanw nickvanw merged commit d38ee62 into main Jan 13, 2026
103 of 106 checks passed
@nickvanw nickvanw deleted the clone-from-donor branch January 13, 2026 22:38
@promptless
Copy link
Contributor

promptless bot commented Jan 13, 2026

📝 Documentation updates detected!

Updated existing suggestion: Add changelog entries for MySQL CLONE support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mattlord mattlord mattlord approved these changes

@nickvanw nickvanw nickvanw approved these changes

@frouioui frouioui Awaiting requested review from frouioui frouioui is a code owner

@harshit-gangal harshit-gangal Awaiting requested review from harshit-gangal

@rohit-nayak-ps rohit-nayak-ps Awaiting requested review from rohit-nayak-ps

@systay systay Awaiting requested review from systay

@timvaillancourt timvaillancourt Awaiting requested review from timvaillancourt

@shlomi-noach shlomi-noach Awaiting requested review from shlomi-noach

Assignees

No one assigned

Labels

Component: Backup and Restore Type: Feature

Projects

None yet

Milestone

v24.0.0

Development

Successfully merging this pull request may close these issues.

4 participants

@maxenglander @mattlord @nickvanw