Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 6 additions & 79 deletions content/en/docs/developer-guides/addon.md
Original file line number Diff line number Diff line change
Expand Up @@ -859,86 +859,9 @@ This architecture graph shows how the coordination between add-on manager and ad

The add-on agent lifecycle can now be managed by the general
`addon-manager` starting from OCM v0.11.0. This is achieved through enhancements
to the `ClusterManagementAddOn` and `ManagedClusterAddOn` APIs.
to the `ClusterManagementAddOn` and `ManagedClusterAddOn` APIs.

1. Install strategy

With the install strategy defined in the `ClusterManagementAddOn` API, users can
configure which clusters the related `ManagedClusterAddon` should be enabled by
referencing the `Placement`. For example, enabling the `helloworld` add-on on
clusters labeled with aws.

```yaml
apiVersion: addon.open-cluster-management.io/v1alpha1
kind: ClusterManagementAddOn
metadata:
name: helloworld
annotations:
addon.open-cluster-management.io/lifecycle: "addon-manager"
spec:
addOnMeta:
displayName: helloworld
installStrategy:
type: Placements
placements:
- name: placement-aws
namespace: default
```

```yaml
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: Placement
metadata:
name: placement-aws
namespace: default
spec:
predicates:
- requiredClusterSelector:
claimSelector:
matchExpressions:
- key: platform.open-cluster-management.io
operator: In
values:
- aws
```

2. Rollout strategy

With the rollout strategy defined in the `ClusterManagementAddOn` API, users can
control the upgrade behavior of the add-on when there are changes in the [supported configurations](#add-your-add-on-agent-supported-configurations).

For example, if the add-on user updates the "deploy-config" and wants to apply
the change to the add-ons to a "canary" [decision group](https://open-cluster-management.io/concepts/placement/#decision-strategy) first.
If all the add-on upgrade successfully, then upgrade the rest of clusters progressively per cluster
at a rate of 25%. The rollout strategy can be defined as follows:

```yaml
apiVersion: addon.open-cluster-management.io/v1alpha1
kind: ClusterManagementAddOn
metadata:
name: helloworld
annotations:
addon.open-cluster-management.io/lifecycle: "addon-manager"
spec:
addOnMeta:
displayName: helloworld
installStrategy:
type: Placements
placements:
- name: placement-aws
namespace: default
configs:
- group: addon.open-cluster-management.io
resource: addondeploymentconfigs
name: deploy-config
namespace: open-cluster-management
rolloutStrategy:
type: Progressive
progressive:
mandatoryDecisionGroups:
- groupName: "canary"
maxConcurrency: 25%
```
More detailed usage of add-on install strategy and rollout strategy refer to the [Add-on lifecycle management](https://open-cluster-management.io/docs/getting-started/installation/addon-management/#add-on-lifecycle-management).

Add-on developers can use addon-framework v0.9.3 and the above versions
to support the scenarios mentioned above.
Expand Down Expand Up @@ -1505,3 +1428,7 @@ namespace, and mount the configmap to the addon agent deployments and daemonsets
`CA_BUNDLE_FILE_PATH` to the file path of the mounted ca bundle. If the addon needs to support the `caBundle` for the
`proxyConfig`, the **addon developer should get the ca bundle from the environment variable** `CA_BUNDLE_FILE_PATH`
to make the agent work with the proxy.

### Add-on template configurations

[Add-on configurations](https://open-cluster-management.io/docs/getting-started/installation/addon-management/#add-on-configurations) provides examples of how to configure add-on templates for different use cases.
118 changes: 84 additions & 34 deletions content/en/docs/getting-started/installation/addon-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ the default add-on configuration. In scenarios where all add-ons have the same
configuration. Only one configuration of the same group and resource can be specified
in the `defaultConfig`.

In the example below, add-ons on all the clusters will use "default-deploy-config" and "default-example-config".
In the example below, add-ons on all the clusters will use "default-deploy-config" and "default-addon-template".

```yaml
apiVersion: addon.open-cluster-management.io/v1alpha1
Expand All @@ -201,23 +201,23 @@ spec:
group: addon.open-cluster-management.io
resource: addondeploymentconfigs
- defaultConfig:
name: default-example-config
name: default-addon-template
namespace: open-cluster-management
group: example.open-cluster-management.io
resource: exampleconfigs
group: addon.open-cluster-management.io
resource: addontemplates
```

### Configurations per install strategy

In `ClusterManagementAddOn`, `spec.installStrategy.placements[].configs` lists the
configuration of `ManagedClusterAddon` during installation for a group of clusters.
For the need to use multiple configurations with the same group and resource can be defined
in this field since OCM v0.15.0. It will override the [Default configurations](#default-configurations)
on certain clusters by group and resource.
in this field since OCM v0.15.0. If [Default configurations](#default-configurations) is defined,
it will override the [Default configurations](#default-configurations) on certain clusters by group and resource.

In the example below, add-ons on clusters selected by `Placement` placement-aws will
use "deploy-config", "example-config-1" and "example-config-2", while all the other add-ons
will still use "default-deploy-config" and "default-example-config".
In the example below, add-ons on clusters selected by `Placement` will
use "override-deploy-config" and "override-addon-template", while all the other add-ons
will still use "default-deploy-config" and "default-addon-template".

```yaml
apiVersion: addon.open-cluster-management.io/v1alpha1
Expand All @@ -235,26 +235,76 @@ spec:
namespace: open-cluster-management
group: addon.open-cluster-management.io
resource: addondeploymentconfigs
- defaultConfig:
name: default-addon-template
namespace: open-cluster-management
group: addon.open-cluster-management.io
resource: addontemplates
installStrategy:
type: Placements
placements:
- name: placement-aws
namespace: default
- name: <placement-name>
namespace: <placement-namespace>
configs:
- group: addon.open-cluster-management.io
resource: addondeploymentconfigs
name: deploy-config
namespace: open-cluster-management
- group: example.open-cluster-management.io
resource: exampleconfigs
name: example-config-1
- name: override-deploy-config
namespace: open-cluster-management
- group: example.open-cluster-management.io
resource: exampleconfigs
name: example-config-2
group: addon.open-cluster-management.io
resource: addondeploymentconfigs
- name: override-addon-template
namespace: open-cluster-management
group: addon.open-cluster-management.io
resource: addontemplates
```

Below are some recommended `Placement` for the install strategy:

- To apply the same configuration across all `ManagedCluster`, use [Default configurations](#default-configurations).

- To apply the configuration to the hub cluster, it must have a specific identifying label or claim, such as local-cluster: "true", for example:

```yaml
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: Placement
metadata:
name: <placement-name>
namespace: <placement-namespace>
spec:
predicates:
- requiredClusterSelector:
labelSelector:
matchLabels:
local-cluster: "true"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is auto-import of hub cluster as managed cluster (local-cluster) something we recently added to OCM-io?
I don't remember this being a default behaviour before.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out! Yes, local-cluster is not added, should not use it directly in doc.

```

- To apply the configuration to the spoke clusters, use the following `Placement`:

```yaml
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: Placement
metadata:
name: <placement-name>
namespace: <placement-namespace>
spec:
predicates:
- requiredClusterSelector:
labelSelector:
matchExpressions:
- key: local-cluster
operator: NotIn
values:
- "true"
# Uncomment the following to install even when clusters are unreachable or unavailable.
# tolerations:
# - key: cluster.open-cluster-management.io/unreachable
# operator: Equal
# - key: cluster.open-cluster-management.io/unavailable
# operator: Equal
```

- To apply the configuration to a specific set of clusters, see [Placement](https://open-cluster-management.io/docs/concepts/content-placement/placement/) for more options.

- To apply the configuration to a single specific cluster, see the next section [Configurations per cluster](#configurations-per-cluster).

### Configurations per cluster

In `ManagedClusterAddOn`, `spec.configs` is a list of add-on configurations.
Expand All @@ -264,7 +314,7 @@ It will override the [Default configurations](#default-configurations) and
[Configurations per install strategy](#configurations-per-install-strategy) defined
in `ClusterManagementAddOn` by group and resource.

In the below example, add-on on cluster1 will use "cluster1-deploy-config" and "cluster1-example-config".
In the below example, add-on on cluster1 will use "cluster1-deploy-config" and "cluster1-addon-template".

```yaml
apiVersion: addon.open-cluster-management.io/v1alpha1
Expand All @@ -274,14 +324,14 @@ metadata:
namespace: cluster1
spec:
configs:
- group: addon.open-cluster-management.io
resource: addondeploymentconfigs
name: cluster1-deploy-config
- name: cluster1-deploy-config
namespace: open-cluster-management
- group: example.open-cluster-management.io
resource: exampleconfigs
name: cluster1-example-config
group: addon.open-cluster-management.io
resource: addondeploymentconfigs
- name: cluster1-addon-template
namespace: open-cluster-management
group: addon.open-cluster-management.io
resource: addontemplates
```

### Supported configurations
Expand All @@ -302,8 +352,8 @@ status:
supportedConfigs:
- group: addon.open-cluster-management.io
resource: addondeploymentconfigs
- group: example.open-cluster-management.io
resource: exampleconfigs
- group: addon.open-cluster-management.io
resource: addontemplates

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here keep addondeploymentconfigs but add addontemplates

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, update to use addondeploymentconfigs and addontemplates as examples, it's more friendly to users.

```

### Effective configurations
Expand All @@ -329,15 +379,15 @@ status:
...
configReferences:
- desiredConfig:
name: cluster1-deploy-config
name: cluster1-addon-template
namespace: open-cluster-management
specHash: dcf88f5b11bd191ed2f886675f967684da8b5bcbe6902458f672277d469e2044
group: addon.open-cluster-management.io
lastAppliedConfig:
name: cluster1-deploy-config
name: cluster1-addon-template
namespace: open-cluster-management
specHash: dcf88f5b11bd191ed2f886675f967684da8b5bcbe6902458f672277d469e2044
lastObservedGeneration: 1
name: cluster1-deploy-config
resource: addondeploymentconfigs
name: cluster1-addon-template
resource: addontemplates
```