Repository navigation
Open
Description
Context
Kubernetes 1.37 adds the EtcdRangeStream feature gate, which the kube rebase enables by default (Beta) in pkg/features/kcp_features.go. Requested in #4514 (comment).
The logical cluster migration currently pages through etcd with plain paginated Get calls. We should investigate whether range streaming can replace or simplify that, and whether the LogicalClusterDump ephemeral resource can be streamed instead of paged.
What exists today
pkg/server/migrationdump/server.go(scanEtcdEntries,nextClusterRange): origin shard scans etcd withkv.Get(..., WithRange, WithLimit(etcdScanLimit))and returns pages inLogicalClusterDump.statuswith acontinuetoken, bounded byspec.limitandspec.maxBytes.pkg/reconciler/migration/logicalclustermigration/datacopy.go(copyPageFromOriginViaHTTP,writeDumpEntries): destination shard requests one dump page per call over HTTP and writes the entries to its own etcd.pkg/reconciler/migration/logicalclustermigration/datacleanup.go(deleteOriginData,scanEtcdKeys): origin cleanup pages through keys withWithLimit(pageSize), then deletes by prefix.- API:
staging/src/github.com/kcp-dev/sdk/apis/migration/v1alpha1/types_logicalclusterdump.go(create-only, non-persisted,spec.continue/limit/maxBytes). - Docs:
docs/content/concepts/sharding/migration.md,pkg/reconciler/migration/logicalclustermigration/doc.go.
To investigate
- Does
EtcdRangeStreamapply to the direct etcd client calls kcp makes here, or only to the kube-apiserver storage layer? Which etcd client/server versions does it need? - Could the dump scan on the origin and the cleanup scan use it, reducing round trips and memory?
- Could
LogicalClusterDumpbe streamed (chunked response or watch-like) instead ofcontinuepages? What would that mean for the v1alpha1 API, which is alpha but already hascontinue/limit/maxBytes? - Interaction with the transformers (decryption on dump, re-encryption on write), the lease handling in
writeDumpEntries, and error/resume behaviour when a stream breaks mid-copy. - Behaviour when the gate is disabled, and a mixed-version shard setup (origin and destination on different versions).
Steps
- Read the upstream
EtcdRangeStreamKEP/implementation and confirm what it covers - Decide whether it applies to the migration scan/copy/cleanup paths
- Prototype the origin dump scan with range streaming and compare memory and latency on a large logical cluster
- Evaluate a streaming
LogicalClusterDumpAPI and its compatibility impact - Write up the findings and, if worthwhile, follow-up implementation issues
- Update
docs/content/concepts/sharding/migration.mdif behaviour changes
Related: PR #4514, comment #4514 (comment)
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
- StatusShow more project fieldsNew
Activity