Skip to content

[release-0.33] Return NotFound to WatchList clients when a shard does not serve the resource - #4546

Closed
kcp-ci-bot wants to merge 1 commit into
kcp-dev:release-0.33from
kcp-ci-bot:cherry-pick-4384-to-release-0.33
Closed

kcp-ci-bot wants to merge 1 commit into
kcp-dev:release-0.33from
kcp-ci-bot:cherry-pick-4384-to-release-0.33

Conversation

@kcp-ci-bot

Copy link
Copy Markdown
Contributor

This is an automated cherry-pick of #4384

/assign ntnn

s
Return NotFound to WatchList clients when a shard does not serve the resource

@kcp-ci-bot kcp-ci-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. dco-signoff: yes Indicates the PR's author has signed the DCO. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 4, 2026
…resource

A watch for a resource that is not served on this shard is answered with an
empty watch that yields no events and stays open, so that a wildcard consumer
aggregating across shards is not wedged by a shard that simply holds none of
these objects.

A WatchList client cannot use such a stream. It treats the watch as its initial
list and waits for the server to close that list with an "initial-events-end"
bookmark before it considers its store synced. An empty watch sends no such
bookmark, and never errors either, so nothing makes the client relist — the
comment's "objects that later appear on this shard are picked up on the next
relist" never happens, because there is no next relist. The client waits
forever:

  reflector.go:327: awaiting required bookmark event for initial events stream,
  no events received for 10.004121875s

That is reachable in a normal deployment. A controller consuming an APIExport
through its virtual workspace starts before any workspace has bound the export,
so the endpoint slice is still empty; when the first binding appears, the
controller connects and establishes a watch for a resource this shard does not
serve yet. Its informers never sync and it never reconciles anything, with no
error anywhere to explain it.

Surface the NotFound to those clients instead: they retry, and pick the
resource up once the shard serves it. Ordinary watches keep the empty watch and
the behaviour that was added for them.

An alternative would be for the empty watch to emit the initial-events-end
bookmark itself, which would let a WatchList client sync against a shard that
holds nothing. That needs a resourceVersion to report as synced-at, and I did
not want to invent one; if there is a right value to use here, that is the
better fix.

Signed-off-by: Waldemar Kindler <waldemar.kindler@think-ahead.tech>
@kcp-ci-bot
kcp-ci-bot force-pushed the cherry-pick-4384-to-release-0.33 branch from c1aacb9 to e99a021 Compare November 4, 2026 12:53

@ntnn ntnn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@kcp-ci-bot kcp-ci-bot added the lgtm Indicates that a PR is ready to be merged. label Nov 4, 2026
@kcp-ci-bot

Copy link
Copy Markdown
Contributor Author

LGTM label has been added.

DetailsGit tree hash: b6badb7b689eb9f5d06b5b8a073e9b43741f405b

@kcp-ci-bot kcp-ci-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 4, 2026
@ntnn

ntnn commented Nov 4, 2026

Copy link
Copy Markdown
Member

/kind bug

@kcp-ci-bot kcp-ci-bot added kind/bug Categorizes issue or PR as related to a bug. and removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Nov 4, 2026
@mjudeikis

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@kcp-ci-bot

Copy link
Copy Markdown
Contributor Author

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mjudeikis, ntnn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kcp-ci-bot

Copy link
Copy Markdown
Contributor Author

@kcp-ci-bot: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kcp-lint e99a021 link true /test pull-kcp-lint
pull-kcp-test-unit e99a021 link true /test pull-kcp-test-unit

Full PR test history

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@mjudeikis

Copy link
Copy Markdown
Contributor

@mjudeikis-bot this backport has issue. make a new pr with fixed unit tests and linting issues

@mjudeikis-bot

Copy link
Copy Markdown

Opened #4547 with the fix: the test called newStorage with three return values, but it returns two on release-0.33, which broke the unit-test and lint builds. Vet, race tests and golangci-lint pass locally.

@mjudeikis mjudeikis closed this Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants