[WIP] ✨ Add clusterctl changes for cross-ns CC ref

[Danil-Grigorev]

What this PR does / why we need it:
This PR allows clusterctl to create a template with a reference to CC located in a different namespace.
This change allows to perform e2e tests in a multi-ns environment.

Added e2e tests to verify cluster-class quick-start scenario and runtime extension scenario in a cross-namespaced context.

The PR is built on top of:

• 🐛 Use namespace of the reference on external.Get #11361
• ✨ Add classNamespace to topology #11352

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #5673

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign sbueringer for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

This PR is currently missing an area label, which is used to identify the modified component when generating release notes.

Area labels can be added by org members by writing /area ${COMPONENT} in a comment

Please see the labels list for possible areas.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[fabriziopandini]

Renamed wip since it builds on a PR not yet merged

[fabriziopandini]

Made first pass, I will get back to this when the PR it builds on top are merged

[fabriziopandini]

Wondering if we can avoid creating a new template by changing cluster-with-topology.yaml so it uses empty string as a default value in case CLUSTER_CLASS_NAMESPACE is not set

[fabriziopandini]

Considering we are trying to keep PR-blocking duration as short as possible, and that the code paths touched by this change are limited and very specific to CC management, would it be possible to add this test on a PR only on demand + on periodic (once it is added to a PR, then it became blocking)

[fabriziopandini]

What about having the cluster in the spec namespace (as we usually do) and create a second namespace for the CC with a name equal to the first namespace + a "cluster-class" suffix; also we probably don't need watches in the CC namespace.

This way when we look at the test, most of the triage work will be consistent with what we usually do, and in case we need to look at the CC, it will be easy to find as it is because it is a namespace with the same root name as everything else

[fabriziopandini]

Just to check if I'm getting this right.
As of today, when we are deploying objects, clusterctl allows to override the namespace set into templates (use case: the user wants to deploy to namespace xyz instead of namespace original).

With this PR we are changing this behaviour, and always preserving the namespace from the template when we are adding the cluster class to the template being deployed (cc preserve: namespace original).

If this is correct, and if I'm not wrong, this change might break users that are deploying a Cluster with ClassNamespace empty, and are expecting that also the CC gets deployed in the target namespace xyz along side with the cluster.

[Danil-Grigorev]

It shouldn’t break these users, since regular process of overriding the namespace of the templated resources is still performed. This new field was never a part of this logic, and the SoT here is the template initially.

This change avoids problems in discovering the existence of the CC in the specified namespace, allowing to template only a Cluster resource.

There is no support in clusterclt to distinguish that some of the resources should go into A namespace, and some other into B, so this multi-namespace setup is intended to happen in a multiple passes.

1. ClusterClass and referenced templates
2. A cluster resource in a different namespace
3. repeat 2 for any namespace without failures on clusterctl side, or any existing template modifications.

[fabriziopandini]

Can we have a similar test when both namespaces are the same, and one where both are empty?

[k8s-ci-robot]

@Danil-Grigorev: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-verify-main	e155375	link	true	/test pull-cluster-api-verify-main

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.