conftest verify does not consider exceptions

[cmorinupgrade]

Hi,

I have the following policy in a file called rbac.rego:

deny_clusterrole_clusterrolebinding_delete[msg] {
	lower(input.kind) == "clusterrole"

	rule := input.rules[_]
	apiGroup := rule.apiGroups[_]
	resource := rule.resources[_]
	verb := rule.verbs[_]

	apiGroup == "rbac.authorization.k8s.io"
	lower(resource) in {"clusterroles", "clusterrolebindings"}
	lower(verb) in {"delete", "deletecollection"}

	msg := sprintf("clusterrole: \"%s\" is not allowed, [input.metadata.name]) 
}

and the following exception (in a separate file called rbac_exception.rego, alongside rbac.rego):

package main

exception[rules] {
	lower(input.kind) == "clusterrole"
	input.metadata.name == "some-app"
	rules := ["clusterrole_clusterrolebinding_delete"]
}

We recently had to add an exception and I would like to write a unit test about that to validate that the exception works as expected.

Here's the test in question:

test_allow_clusterrole_delete_for_exceptions {
	cfg := parse_config("yaml", `
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: some-app
  rules:
  - apiGroups:
    - "rbac.authorization.k8s.io"
    resources:
    - clusterroles
    - roles
    verbs:
    - get
    - create
    - update
    - delete
    - list
    - watch
  `)
	empty(deny_clusterrole_clusterrolebinding_delete) with input as cfg
}

Using the following command does not seem to take the exception into consideration (always fail):

➜ conftest verify --no-color -p kubernetes
FAIL - kubernetes/rbac_test.rego -  - data.main.test_allow_clusterrole_delete_for_exceptions

84 tests, 83 passed, 0 warnings, 1 failure, 0 exceptions, 0 skipped

Any help would be much appreciated.

Thank you

[anderseknert]

I'm not sure, but it seems only conftest test allows exceptions, as described in the docs:

❯ conftest verify config.yaml
0 tests, 0 passed, 0 warnings, 0 failures, 0 exceptions, 0 skipped

❯ conftest test config.yaml
EXCP - config.yaml - main - data.main.exception[_][_] == "foo"

1 test, 0 passed, 0 warnings, 0 failures, 1 exception

The docs could certainly be improved to explain that, and if exceptions don't apply to conftest verify it would perhaps be good to remove that from the output of that command. I'd suggest filing an issue in the conftest repo about this.

[cmorinupgrade]

Will do. Thanks @anderseknert 👍