Safe subdirectory based CNI chain configuration loading by dougbtv · Pull Request #1419 · k8snetworkplumbingwg/multus-cni

dougbtv · 2025-04-09T18:46:07Z

Implements a method for loading CNI plugins in a chain using a cnilib v1.3.0 feature for safe subdirectory based configuration loading, as introduced in containernetworking/cni#1052

See also, this asciinema demo of the functionality: https://asciinema.org/a/LKwyxyxdafgps5BXneME7ZMJE

coveralls · 2025-04-09T18:51:45Z

coverage: 52.959% (-2.5%) from 55.425%
when pulling 4104fea on dougbtv:rebase5-cni-subdirectory-chain
into 55ef3b1 on k8snetworkplumbingwg:master.

dougbtv · 2025-04-09T19:29:19Z

 }

-func conflistAdd(rt *libcni.RuntimeConf, rawnetconflist []byte, multusNetconf *types.NetConf, exec invoke.Exec) (cnitypes.Result, error) {
+func conflistAdd(rt *libcni.RuntimeConf, rawnetconflist []byte, cniConfList *libcni.NetworkConfigList, multusNetconf *types.NetConf, exec invoke.Exec) (cnitypes.Result, error) {


TODO: Check that this works without conflists

So, I think it works as intended without conflists... If you don't have a conflist to begin with -- you can't chain on it.

But in that case, you can use AuxiliaryCNIChainName and have that as a place to chain instead.

dougbtv · 2025-04-09T19:32:04Z

+	return delegateConf, nil
+}
+
+// !bang


remove that !bang

bpickard22 · 2025-04-11T18:15:32Z

 - `"logLevel"`: the logging level for the multus daemon logs.
 - `"logToStderr"`: enable this to have the daemon multus logs echoed to stderr
 as well. By default, it is disabled.
+- `"auxiliaryCNIChainName"`: set a value to executed chained cni configurations from disk in an auxiliary CNI chain (see details in [configuration.md](configuration.md))


nit: replace executed with execute

got it! thx

bpickard22 · 2025-04-11T18:16:04Z

 In thick plugin case, delegate CNI plugin is executed by multus-daemon from Pod, hence if the delegate CNI requires resources in container host, for example unix socket or even file, then CNI plugin is failed to execute because multus-daemon runs in Pod. Multus-daemon supports "chrootDir" option which executes delegate CNI under chroot (to container host).

 This configuration is enabled in deployments/multus-daemonset-thick.yml as default.
+


nit: whitespace

got it, should just be newline now

bpickard22 · 2025-04-11T18:17:12Z

-				confList, err := libcni.ConfListFromFile(netname)
+				confList, err := LoadChainedPluginsFromFile(netname)
 				if err != nil {
 					return nil, resourceMap, logging.Errorf("error loading CNI conflist file %s: %v", netname, err)


i think we should change the log to reflect the change from loading in conf files to loading in plugins

good call, I put a few debug output lines in LoadChainedPluginsFromFile so that one can see that it's happening if it matters at that time.

bpickard22 · 2025-04-11T18:18:12Z

+
 			}
-			delegate, err := types.LoadDelegateNetConf(configBytes, nil, "", "")
+


how would we get to this point? i remember you had a brief explanation when we talkied, might be good to add a comment here

yeah! good question, so, you can definitely get there, say, if you provide a file path that is a .conf -- which is the case for say, ovn-kubernetes. In that case, you can't chain onto a .conf, only a .conflist, in which case -- you must use the auxiliaryCNIChain to add chaining. So if you use ovn-k as your primary CNI, you'll have to configure multus to use the aux CNI chain, which will then load the subdirectory chain.

…ation loading. Removes the it `fails to execute confListDel given no 'plugins' key"` test. This test no longer fails after libcni version 1.2.3. It probably shouldn't failduring a DEL action as it is, we want the least error prone path. The GC test now uses both cni.dev attachment formats. Uses both attachment formats as per containernetworking/cni#1101 for GC's cni.dev/valid-attachments & cni.dev/attachments

Adds a test for plain subdirectory chaining and also using passthru CNI with auxiliaryCNIChainName

A delegated plugin that requires configuration fails for del and check calls. Porting back the code added to the add call in k8snetworkplumbingwg#1419 resolves the issue. Fixes k8snetworkplumbingwg#1449 Signed-off-by: Jens Erat <email@jenserat.de>

[bump] Bumps to libcni v1.3.0

fa3c7cf

github-advanced-security AI found potential problems Apr 9, 2025

View reviewed changes

Comment thread pkg/k8sclient/k8sclient.go Fixed

Comment thread pkg/types/conf.go Fixed

dougbtv force-pushed the rebase5-cni-subdirectory-chain branch from fa49747 to d7a450c Compare April 9, 2025 19:02

dougbtv commented Apr 9, 2025

View reviewed changes

Comment thread pkg/types/conf.go Outdated

return delegateConf, nil

}

// !bang

Copy link
Copy Markdown

Member Author

dougbtv Apr 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove that !bang

dougbtv force-pushed the rebase5-cni-subdirectory-chain branch 2 times, most recently from 31b08b3 to 591f6d1 Compare April 9, 2025 20:32

github-advanced-security AI found potential problems Apr 9, 2025

View reviewed changes

Comment thread pkg/k8sclient/k8sclient.go Dismissed

Comment thread pkg/k8sclient/k8sclient.go Dismissed

dougbtv force-pushed the rebase5-cni-subdirectory-chain branch 9 times, most recently from 085c8a7 to f41b35a Compare April 10, 2025 20:59

bpickard22 reviewed Apr 11, 2025

View reviewed changes

dougbtv added 2 commits April 15, 2025 15:53

Subdirectory CNI chain loading e2e tests

4104fea

Adds a test for plain subdirectory chaining and also using passthru CNI with auxiliaryCNIChainName

dougbtv force-pushed the rebase5-cni-subdirectory-chain branch from f41b35a to 4104fea Compare April 15, 2025 19:53

bpickard22 merged commit 4517063 into k8snetworkplumbingwg:master Apr 16, 2025
25 checks passed

SaberSHO mentioned this pull request Apr 18, 2025

Latest change to Thick Image deployment.yaml breaks Multus on Talos Cluster #1422

Closed

JensErat mentioned this pull request Oct 11, 2025

Delegated CNI conflist passed with add, but not check/del #1449

Open

JensErat mentioned this pull request Oct 11, 2025

Pass delegated plugin conflist also for del/check calls #1450

Open

		In thick plugin case, delegate CNI plugin is executed by multus-daemon from Pod, hence if the delegate CNI requires resources in container host, for example unix socket or even file, then CNI plugin is failed to execute because multus-daemon runs in Pod. Multus-daemon supports "chrootDir" option which executes delegate CNI under chroot (to container host).

		This configuration is enabled in deployments/multus-daemonset-thick.yml as default.


		}
		delegate, err := types.LoadDelegateNetConf(configBytes, nil, "", "")

Conversation

dougbtv commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coveralls commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dougbtv commented Apr 9, 2025 •

edited

Loading

coveralls commented Apr 9, 2025 •

edited

Loading