Skip to content

[3- De-Rooting] Move serviceaccount of root shard - #4494

Merged
kcp-ci-bot merged 4 commits into
kcp-dev:mainfrom
mjudeikis:de.root.the.root.sa.accounts
Nov 3, 2026
Merged

kcp-ci-bot merged 4 commits into
kcp-dev:mainfrom
mjudeikis:de.root.the.root.sa.accounts

Conversation

@mjudeikis

Copy link
Copy Markdown
Contributor

Summary

Must merge after #4493
Moves root shards off root shard so making root shard "just another shard"

What Type of PR Is This?

Related Issue(s)

Fixes #

Release Notes

NONE

@kcp-ci-bot kcp-ci-bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the DCO. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Oct 26, 2026
@mjudeikis mjudeikis changed the title [wip] Move serviceaccount of root shard [wip][3- De-Rooting] Move serviceaccount of root shard Oct 26, 2026
@kcp-ci-bot kcp-ci-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 30, 2026
@mjudeikis
mjudeikis force-pushed the de.root.the.root.sa.accounts branch from 8e0e22c to ce14f56 Compare November 2, 2026 06:12
@kcp-ci-bot kcp-ci-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 2, 2026
@mjudeikis mjudeikis changed the title [wip][3- De-Rooting] Move serviceaccount of root shard [3- De-Rooting] Move serviceaccount of root shard Nov 2, 2026
@kcp-ci-bot kcp-ci-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 2, 2026
@mjudeikis

Copy link
Copy Markdown
Contributor Author

@mjudeikis-bot review this

@mjudeikis-bot mjudeikis-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two verified Warnings and two Notes; no Critical found. The routing itself (index lookup, then shards credential, then the hosting shard) matches how the other proxy clients are built and is scoped per logical cluster.

Not anchored

  • Warning: no test covers service account authentication through the front proxy. pkg/proxy/serviceaccounts_test.go covers only the lookup type against a fake shard. Nothing builds Authentication.ApplyTo with a ServiceAccountLookup, and cmd/sharded-test-server/frontproxy.go never passes --service-account-key-file, so no e2e reaches this path. The wiring of ServiceAccountTokenGetter and SecretsWriter is therefore unexercised. A test that sends a signed legacy token or bound token through the authenticator against two fake shards would pin it.

Checked

  • Read beyond the diff: pkg/proxy/index/index_controller.go (Lookup, BaseURL, Start), pkg/proxy/server.go (NewServer, Run, readyz wiring, /metrics mux), pkg/proxy/metrics_authorizer.go, pkg/proxy/config.go, the kcp-dev/kubernetes serviceaccount.NewClusterGetterFromClient (the old path), cmd/sharded-test-server/frontproxy.go, and docs/content/setup/production/metrics.md.
  • Correctness: the old informer factory in authentication.go was never started, so its listers were always empty and every lookup was already a live GET. The new code adds no extra round trips. Findings are below. An unknown cluster maps to NotFound and a missing index to ServiceUnavailable (see Note on 401).
  • Concurrency: clientFor guards index and clients with mu; no lock-order issue. Findings: the clients map is never pruned (Note).
  • kcp specifics: every call goes through client.Cluster(cluster.Path()) after index.Lookup. A wildcard or path-form cluster is rejected in routedSecrets.Cluster. The shards credential and the identity round tripper are preserved through rest.CopyConfig. No cross-workspace read without a lookup.
  • API and storage: no CRD or stored-format change. --root-kubeconfig stays as a deprecated flag; see the Warning on startup.
  • Security: the :root delegated SubjectAccessReview for the front-proxy /metrics is removed. This matches the group-based text in metrics.md, but the PR carries release-note NONE (Note). No credential reaches an error string.
  • Tests: the new unit test covers index-missing, known cluster, unknown cluster, patch and list-unsupported. The Authentication wiring and Validate change have no test (Warning above).
  • Build/test: go build ./pkg/proxy/... ./cmd/... OK; go vet ./pkg/proxy/... OK; go test ./pkg/proxy/ ./pkg/proxy/options/... OK. I did not run the linter or any e2e: the sharded e2e needs a full multi-shard setup and no existing e2e enables proxy SA auth.

Comment thread pkg/proxy/options/options.go Resolved
Comment thread pkg/proxy/serviceaccounts.go Resolved
Comment thread pkg/proxy/server.go Resolved
Comment thread pkg/proxy/server.go Resolved
@mjudeikis

Copy link
Copy Markdown
Contributor Author

/retest

@mjudeikis
mjudeikis force-pushed the de.root.the.root.sa.accounts branch from a8652cc to 8625ed9 Compare November 2, 2026 08:01
@kcp-ci-bot kcp-ci-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 2, 2026
@mjudeikis
mjudeikis force-pushed the de.root.the.root.sa.accounts branch from 8625ed9 to 57c7c88 Compare November 2, 2026 11:12
@mjudeikis

Copy link
Copy Markdown
Contributor Author

/retest

Comment on lines +50 to +56
type ShardRoutedServiceAccounts struct {
shardsConfig *rest.Config

mu sync.Mutex
index shardIndex // nil until the index controller exists
clients map[string]shardClient // by shard name
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any caching at all? Wouldn't that mean that every SA request would be multiple GETs to the shard behind it?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The clients are cached but not the results. I think that would be a usecase for the shardlookup package.

Comment thread pkg/proxy/serviceaccounts_test.go Outdated
Comment on lines +42 to +46
// fakeIndex maps logical clusters to shards and shards to base URLs.
type fakeIndex struct {
shards map[logicalcluster.Name]string
urls map[string]string
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we already have a fake index for the front-proxy? I think its private in another package?
I'd rather extract that/make that available then reimplementing everywhere where we use an fake index for testing.

Comment on lines +326 to +329
synced, found := c.shardInformersSynced[shard.Name]
if !found {
return fmt.Errorf("informers for shard %q are not started", shard.Name)
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if a known shard is down?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably at least exclude shards that are marked for maintenance etcpp instead of taking all shards.

@kcp-ci-bot kcp-ci-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 3, 2026
@mjudeikis
mjudeikis force-pushed the de.root.the.root.sa.accounts branch from 2adb559 to b9d45e5 Compare November 3, 2026 14:03
@kcp-ci-bot kcp-ci-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 3, 2026

@ntnn ntnn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@kcp-ci-bot kcp-ci-bot added the lgtm Indicates that a PR is ready to be merged. label Nov 3, 2026
@kcp-ci-bot

Copy link
Copy Markdown
Contributor

LGTM label has been added.

DetailsGit tree hash: c91249d197785dd4d8d3f733734f1e2460fc5bf0

@kcp-ci-bot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ntnn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kcp-ci-bot kcp-ci-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 3, 2026
@kcp-ci-bot
kcp-ci-bot merged commit eb94397 into kcp-dev:main Nov 3, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API lgtm Indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants