CNF-15663: Full DU profile example by irinamihai · Pull Request #313 · openshift-kni/oran-o2ims

irinamihai · 2024-11-15T23:25:30Z

No description provided.

openshift-ci-robot · 2024-11-15T23:25:33Z

@irinamihai: This pull request references CNF-15663 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.18.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

irinamihai · 2024-11-15T23:25:43Z

/hold cleanup

irinamihai · 2024-11-18T18:56:15Z

/unhold

browsell · 2024-11-17T20:50:57Z

This should be a default

+1 -- or going a bit further I think this is static content that likely doesn't need to be part of the defaults either.

It has been agreed that this will be locked in the new version of the ClusterLogForwarder, currently WIP under OCPBUGS-44518, so it will be removed from this ClusterTemplate.

browsell · 2024-11-17T20:51:17Z

This should be a default

By default, do you mean have them directly in the Policy Generator and not expose them in the policyTemplateParameters?

No, these would be set in the default configmap vs being passed in by the client

As proposed above I think we should narrow this down to just the additional labels. One question I have is whether the user/orchestrator would add one or more labels which are cluster specific (like a higher level cluster identifier, etc)? In that case would the labels (or at least one label) need to be part of this schema?

The filters are also going to be partially locked in in the ClusterLogForwarder source-cr under OCPBUGS-44518. Yes, these labels will be set in the ClusterInstance defaults ConfigMap, but we also need a way for them to reach the ConfigMap used by the ACM PGs, so they need to also be included in the policyTemplate defaults ConfigMap.

browsell · 2024-11-17T20:51:30Z

browsell · 2024-11-17T20:51:37Z

browsell · 2024-11-17T20:52:07Z

browsell · 2024-11-18T19:23:00Z

Not really filters, additional metadata labels

Do we want to narrow the templating down to just the additional labels, ie the user configures only the value for openshiftLabels?

browsell · 2024-11-18T19:29:53Z

Remove, see comment above

browsell · 2024-11-18T19:30:03Z

browsell · 2024-11-18T20:18:45Z

browsell · 2024-11-18T20:19:00Z

lack

Generally speaking, if this is something that will need to be kept in-sync with the cnf-features-deploy repo, perhaps it's worth engineering a way to automatically synchronize them or generate one from the other.

lack · 2024-11-19T03:32:22Z

Should we allow customization of all of this? Or just the Kafka url?

lack · 2024-11-19T03:40:53Z

We shouldn't override this section, but rely on the source-crs original value

+1
This is also missing the module_blacklist=irdma

I will remove this section to keep the defaults from the source-cr. I kept it from a previous configuration, but I realized it's not matching the 4.17 configuration.

lack · 2024-11-19T03:42:56Z

Suggested change

machineConfigPoolSelector:

pools.operator.machineconfiguration.openshift.io/master: ""

nodeSelector:

node-role.kubernetes.io/master: ''

machineConfigPoolSelector:

$patch: replace

pools.operator.machineconfiguration.openshift.io/master: ""

nodeSelector:

$patch: replace

node-role.kubernetes.io/master: ''

And then we don't need the SetSelector cr variant any more.

(Repeat for *-SetSelector.yaml elsewhere in this file!)

This should be fine now since we also have support for openapi schemas!

lack · 2024-11-19T03:45:00Z

We shouldn't override these; the source-crs has the right values.

lack · 2024-11-19T03:52:01Z

i don't think any of this is needed any more since the SiteConfig added cpuPartitioningMode: AllNodes in 4.14

browsell · 2024-11-19T11:07:35Z

Wouldn't this be in the source cr ?

The selector can't be, because depending on whether you're deploying SNO or MNO the source CR may need master or worker.

In this model, the cluster template would only be used for SNO, a MNO would have a different one,

For SNO we should be able to use either master or worker here since the node has both labels. If we use worker is it valid for all topologies?

browsell · 2024-11-19T11:08:09Z

Do we need to override the default profile? 🤔 The ztp git example is just using the default profile values.

No need to override the source cr here

browsell · 2024-11-19T11:08:19Z

browsell · 2024-11-19T11:08:31Z

browsell · 2024-11-19T11:08:43Z

browsell · 2024-11-19T11:09:13Z

Missxiaoguo · 2024-11-20T15:13:08Z

By default, observability is not enabled. I think we should just use the default values from source-cr.

we should enable observability.

I have referred to the story IanM has pointed out (CNF-13398) and we have to merge our desired configuration for reducing the monitoring footprint with the default observability config so that our configuration does not get overridden.
I will include the new manifest in the following patch.

Missxiaoguo · 2024-11-20T15:18:04Z

Is this DU profile based on the OCP 4.17? I wonder if adding a comment to mention that might be helpful.

Missxiaoguo · 2024-11-20T15:20:52Z

Do we want to use disconnected registry as example?

imiller0 · 2024-11-20T20:46:50Z

Is this intentional?

Yes, just to showcase how to add extra annotations, but I think it can be removed from the full DU profile example and kept in our other examples.

imiller0 · 2024-11-20T21:15:09Z

Reference is now UEFISecureBoot.

imiller0 · 2024-11-20T21:16:42Z

For IBU there is a need for a separate partition for /var/lib/containers. Should this example set that up as well?

yes, good catch

I will update to use the config from the SiteConfig 4.17 examples then.

imiller0 · 2024-11-20T21:19:59Z

Is there an intent to include ports eth0 and eth1 in this bond, or other ports in it?

imiller0 · 2024-11-20T21:23:06Z

Do we want to narrow the templating down to just the additional labels, ie the user configures only the value for openshiftLabels?

imiller0 · 2024-11-20T22:20:17Z

As noted above we should use more fixed content in the source CR (ie be more opinionated) and allow the user to override the URL and labels.

imiller0 · 2024-11-20T22:20:28Z

Already in source CR

We needed to specify it if we use $patch: replace.
This whole manifest will be reworked anyway with the new ClusterLogForwarder source-cr.

imiller0 · 2024-11-20T22:21:30Z

These are not in the reference. Is their addition intentional?

imiller0 · 2024-11-20T22:22:01Z

Missing:
group.ice-dplls=0:f:10:*:ice-dplls.*

imiller0 · 2024-11-20T22:23:49Z

For all of these we should not repeat any of the content already in the source CR.

Hmm, there is no content locked in the source-crs...
SriovNetworkNodePolicy, SriovNetwork.

imiller0 · 2024-11-20T22:31:55Z

Is there a reason this is in its own policy? This creates more policies than are necessary. Consider combining with the next policy as "baseline config"

Actually, I think this can be included in the v4-config-policy.

Missxiaoguo · 2024-11-20T23:33:02Z

Could sriov configs be included in the v4-config-policy? I don't see the reason why they couldn't be🤔 .

browsell · 2024-11-21T04:28:40Z

Can we remove the bond, bonding is going to be very rare for this use case

irinamihai · 2024-12-05T00:16:51Z

/hold
Final cleanup

irinamihai · 2024-12-05T18:21:29Z

/unhold

bartwensley · 2024-12-05T20:57:34Z

Hmm... how would this work if we had more than one cluster template that needed to enable observability? Wouldn't this result in a conflicting binding?

This one has to be a one time policy per namespace. We could have a directory called common-<namespace> under sno-ran-full-du and include there all the resources common for the ClusterTemplates in a certain namespace. WDYT, @bartwensley?

My concern was that the ManagedClusterSetBinding refers to "open-cluster-management-observability" which doesn't include anything specific to the cluster template. So if you had two different cluster templates, and they each contained this same ManagedClusterSetBinding, wouldn't that be a conflict? So don't we need to include the cluster template name in this namespace? Apologies if I'm not explaining this well (or if it makes no sense).

@bartwensley , yes, it makes sense. I've thought about it and I pushed a new patchset with a new approach.
We could use the following directory structure:

policytemplates/ common/ acm-pg-observability.yaml msc-observability.yaml source-cr-observability.yaml version_4.X.Y/ version_4.X.Y+1/ ...

In the new patchset, I've used object-templates-raw in the updated source-cr such that we range over the namespaces where we create ORAN policies. The only drawback is that we need to include those namespaces manually. The alternative would have been to have one ACM PG for each ztp-<clustertemplate-namespace>, so I think the current proposal is still cleaner.

I will check with ACM folks if there is any way for us to automatically get the namespaces and I can update that in a future PR.

What do you think?

Thanks Irina - looks good.

browsell · 2024-12-06T20:29:15Z

where did this come from. We do not want all of these enabled on the managed cluster

These come from values set by ACM observability. On the hub we (typically) disable ACM's ability to write this configmap because of conflicting settings for this "yaml in a string" value, but we merged our content with what ACM sets. The collectors entries below each have a boolean enable/disable value. With this setting only netclass and netdev are enabled which matches the state when the configmap is removed entirely.

For reference, the node-exporter pod command line looks like this with the nodeExporter config in this PR.

- --no-collector.wifi - --collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run/k3s/containerd/.+|var/lib/docker/.+|var/lib/kubelet/pods/.+)($|/) - --collector.netclass.ignored-devices=^(veth.*|[a-f0-9]{15}|enP.*|ovn-k8s-mp[0-9]*|br-ex|br-int|br-ext|br[0-9]*|tun[0-9]*|cali[a-f0-9]*)$ - --collector.netdev.device-exclude=^(veth.*|[a-f0-9]{15}|enP.*|ovn-k8s-mp[0-9]*|br-ex|br-int|br-ext|br[0-9]*|tun[0-9]*|cali[a-f0-9]*)$ - --collector.cpu.info - --collector.textfile.directory=/var/node_exporter/textfile - --no-collector.btrfs - --runtime.gomaxprocs=0 - --no-collector.cpufreq - --no-collector.tcpstat - --collector.netdev - --collector.netclass - --collector.netclass.netlink - --no-collector.buddyinfo - --no-collector.mountstats - --no-collector.ksmd - --no-collector.processes - --no-collector.systemd

imiller0 · 2024-12-06T21:12:16Z

Is the path right? I note that the CR is in sno-ran-full-du-profile directory.

Uh-oh... I actually renamed the directories to be sno-ran-full-du. Thank you.

irinamihai · 2024-12-17T21:42:00Z

/retest-required

bartwensley · 2024-12-19T13:48:58Z

/lgtm

browsell · 2024-12-19T14:15:25Z

/approve

openshift-ci · 2024-12-19T14:15:36Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: browsell

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [browsell]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

irinamihai requested a review from Missxiaoguo November 15, 2024 23:25

openshift-ci Bot requested review from pixelsoccupied and sudomakeinstall2 November 15, 2024 23:25

openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 15, 2024

openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 18, 2024

irinamihai requested review from bartwensley, browsell, imiller0, lack and pixelsoccupied and removed request for pixelsoccupied and sudomakeinstall2 November 18, 2024 18:57

browsell reviewed Nov 18, 2024

View reviewed changes

lack reviewed Nov 19, 2024

View reviewed changes

browsell reviewed Nov 19, 2024

View reviewed changes

bartwensley reviewed Nov 20, 2024

View reviewed changes

Comment thread docs/cluster-configuration.md Outdated

Comment thread docs/samples/git-setup/policytemplates/version_4.Y.Z/kustomization.yaml Outdated

Missxiaoguo reviewed Nov 20, 2024

View reviewed changes

imiller0 reviewed Nov 20, 2024

View reviewed changes

Missxiaoguo reviewed Nov 20, 2024

View reviewed changes

browsell reviewed Nov 21, 2024

View reviewed changes

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 27, 2024

irinamihai force-pushed the full-du-profile branch 2 times, most recently from 4598a0b to 071bcc5 Compare December 5, 2024 00:15

openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 5, 2024

irinamihai force-pushed the full-du-profile branch 2 times, most recently from 56c7b70 to fa33b9e Compare December 5, 2024 18:17

openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 5, 2024

irinamihai requested review from Missxiaoguo, bartwensley, browsell, imiller0 and lack December 5, 2024 18:21

bartwensley reviewed Dec 5, 2024

View reviewed changes

irinamihai force-pushed the full-du-profile branch from fa33b9e to 8270259 Compare December 5, 2024 23:12

browsell reviewed Dec 6, 2024

View reviewed changes

imiller0 reviewed Dec 6, 2024

View reviewed changes

irinamihai force-pushed the full-du-profile branch from 8270259 to df64aaf Compare December 9, 2024 21:01

irinamihai force-pushed the full-du-profile branch from df64aaf to 39378d0 Compare December 17, 2024 21:05

CNF-15663: Full DU profile example

51c608e

irinamihai force-pushed the full-du-profile branch from 39378d0 to 51c608e Compare December 17, 2024 21:31

openshift-ci Bot assigned bartwensley Dec 19, 2024

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Dec 19, 2024

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 19, 2024

openshift-merge-bot Bot merged commit 7c9a207 into openshift-kni:main Dec 19, 2024

irinamihai deleted the full-du-profile branch January 9, 2025 20:34

-          machineConfigPoolSelector:
-            pools.operator.machineconfiguration.openshift.io/master: ""
-          nodeSelector:
-            node-role.kubernetes.io/master: ''
+          machineConfigPoolSelector:
+            $patch: replace
+            pools.operator.machineconfiguration.openshift.io/master: ""
+          nodeSelector:
+            $patch: replace
+            node-role.kubernetes.io/master: ''

Conversation

irinamihai commented Nov 15, 2024

Uh oh!

openshift-ci-robot commented Nov 15, 2024 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

irinamihai commented Nov 15, 2024

Uh oh!

irinamihai commented Nov 18, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

browsell Nov 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lack left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

openshift-ci-robot commented Nov 15, 2024 •

edited by openshift-ci Bot

Loading

browsell Nov 19, 2024 •

edited

Loading