feat(otelcolLogs): Split kubelet from systemd logs with a separate feature flag

[jagan2221]

Split kubelet logs from systemd logs and provided a separate feature flag(sumologic.logs.kubelet.enabled) to toggle kubelet logs collection.

Why?
Currently, if sumologic.logs.systemd.enabled flag We have already mentioned in docs that kubelet logs can be configured separately, but haven't provided provision for that.

Ref JIRA -> https://sumologic.atlassian.net/browse/OSC-784

Checklist

• Changelog updated or skip changelog label added
• Documentation updated
• Template tests added for new features
• Integration tests added or modified for major features

[chan-tim-sumo]

just curious, is there a reason to move kubelet.service to the end? looks like this was in alphabetical order 🤔

[jagan2221]

https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/3845/files#diff-bd3221290637efd3855e2de7717869ccb4e6eb4cf2dcc177729dd61e9cb2fe06R121

In config yaml since we have splitted systemd and kubelet with separate flags, all systemd units come first and kubelet after those. Previously since kubelet was as subset of systemd, we have kept systemd units in alphabetical order.
@chan-tim-sumo

[ccressent]

What is it that you cannot achieve using the existing sumologic.logs.kubelet.* knobs?

To "disable kubelet independently of systemd", as the JIRA ticket mentions, why not, for example, use sumologic.logs.kubelet.otelcol.extraProcessors and create an appropriate filter that drops everything?

[jagan2221]

https://sumologic.atlassian.net/browse/OSC-784

I feel that that would be like a workaround , we already have suggested other workaround to the customer who raised this,
#3168 (comment) - This thread has customer's concern using the workaround and Dominik mentioning about separating kubelet in future.

https://help.sumologic.com/docs/send-data/kubernetes/collecting-logs/#kubelet-logs
Also here we are mentioning the significance of kubelet logs and that can be configured separately as well.
@ccressent

[rnishtala-sumo]

@jagan2221 I think for changes like these we need new integration tests since this is change to a major feature (logs collection)