Proposal for opa version 0.43.1

[mlajkim]

Hello,

Our team has been using opa and kube-mgmt for a long time with the following fixed versions:

• opa 0.43.0
• kube-mgmt 7.0.6

Since both versions have vulnerabilities, we want to update the package with no-found-vulnerabilities to meet our standards.
Yet it seems like there are some changes in:

• Rego grammar
• opa configurations
• etc that we are not aware of

between the current image 0.43.0 and the latest image 1.0.0.It is our fault that we haven't upgraded periodically, but yeah our team was a bit busy with other stuff.

Correct me if I am wrong, but our system indicates that every opa version below 0.64.0 has vulnerabilities.
So if we upgrade to an already-released version, it will most likely be 0.64.0.

Upgrading from 0.43.0 and 0.64.0 will not only take us some time to gradually upgrade
but also since we need to modify our Rego policies, there's a risk that these changes could unknowingly disrupt our production environments.

therefore, if possible, we would like to maintain our current policies while addressing the vulnerability issue with the following steps:

1. We create a new branch named 0.43.1-release based on 0.43.0.
2. We make a PR that fixes the vulnerabilities for 0.43.0-fix-vuln based on the branch 0.43.1-release.
3. Have the OPA team review/merge the PR into the 0.43.1-release branch.
4. Have the OPA team release 0.43.1 as a patch version.

This way, we can temporarily upgrade our internal OPA usage from 0.43.0 to 0.43.1 and later upgrade to the latest version 1.0.0 once we've completed some internal tasks we have in our organization.

Do you think this approach is feasible? Is there anything else I might be overlooking?

We would greatly appreciate your assistance with this matter.
Thank you in advance for considering our request.

Have a lovely day.

[charlieegan3]

Hi @mlajkim, thank you for starting the discussion. I’m sorry to hear about your situation and will do my best to assist.

First, in case you were unaware, there is a --v0-compatible mode for OPA 1.0. You can find details here. This is the recommended approach for those needing a quick upgrade without modifying policies.

That said, I encourage you and the team to consider this point:

Upgrading from 0.43.0 and 0.64.0 will not only take us some time to gradually upgrade.

With the tools available now, the process may be less arduous than you expect.

Using an OPA 1.0 binary locally, you can run:
• opa check --v0-v1 --strict to identify issues that are now compiler errors.
• opa fmt --write --v0-v1 to automatically upgrade Rego to the new syntax.
• regal lint to use the Regal linter for detecting additional bugs.

More details here: https://www.styra.com/blog/renovating-rego/

If you haven’t already, consider adding test coverage to confirm the behavior remains consistent before and after the process.

This effort is likely more valuable than delaying the upgrade. You’ll not only benefit from modernized Rego but also from the significantly faster and improved OPA, which has seen substantial progress over the past two years.

If you choose the 0.43.0-fix-vuln path, note that this will need to be done in a private fork, as it’s not something we can support in the public OPA repository.

[mlajkim]

@charlieegan3
Hi, thank you for the kind response!

Our team will begin the operation check for OPA's "v0 backwards compatibility" as you've shared starting on Jan 24 JST.
I will post some questions in this discussion if we have any during the operation check.

Thank you in advance.

[mlajkim]

@anderseknert @charlieegan3
Hi,

We've concluded that the --skip-known-schema-check flag indeed resolves the problem. However, as you mentioned, we also believe it's essential to address the underlying issues, such as error handling.

The NullReferenceError occurs because of the following logic:

default unverified_jwt = [null, null]
unverified_jwt = io.jwt.decode(jwt)

This code relies on io.jwt.decode. Our Rego policy then attempts something like:

unverified_jwt[0].kid == json.unmarshal(jwks_cached).keys[_].kid

This operation causes the NullReferenceError if json.unmarshal(jwks_cached) is null. To fix this, we'll modify the code as follows:

unverified_jwt = decoded_jwt {
    decoded_jwt := io.jwt.decode(jwt)
} else = [{ "kid": null }, { "exp": null }]

This ensures that the keys kid or exp are always present in the JWT.

I'm still puzzled about why the restarts occur. Even when it "restarts," it loads afterward successfully (when it is still a initial booting stage). The results are inconsistent, making it difficult to pinpoint the exact cause of the flakiness.

We also sometimes get the following log, but again randomly:

/etc/opa/policy/authorization_policy.rego:151: rego_type_error: io.jwt.decode_verify: invalid argument(s)
	have: (string, null, ???)
	want: (jwt: string, constraints: object[string: any], output: array<boolean, object[any: any], object[any: any]>)

Is there any docs/tutorials that I should learn so that I can truly understand the issue I am facing? This week I am fully committed to address this issue.
Thanks in advance.

p.s)
I believe the feature is introduced in the following PR: open-policy-agent/opa#6141

[mlajkim]

Have restarted 50 times+. The init is a bit flaky with the following result:

[charlieegan3]

The type error you are seeing, I think, is unrelated to the restarts. Can you look into the logging from the containers nearer the times OPA is being restarted? If you are using K8s, the pod status might shed some light too on what's happening, it could be getting OOMKilled for example.

[charlieegan3]

You might find the reference at https://github.com/StyraInc/lib.jwt a helpful reference when decoding jwts correctly.

[mlajkim]

@charlieegan3
Due to other priorities, I’ll move on from investigating the flaky container restarts for now. However, I’ve fixed the policy to make it more robust, so it no longer restarts unexpectedly (without the flag --skip-known-schema-check)

Thanks for the amazing support! If you don’t mind, I’d like to start another discussion on the restart issue when we have time

[mlajkim]

Closed as mentioned here: https://github.com/orgs/open-policy-agent/discussions/651?notification_referrer_id=NT_kwDOAyyqzrQxNDMwMDQ0ODkyODo1MzI1ODk1OA#discussioncomment-12394635

Thank you once again for the amazing support. I was able to fix our internal rego policy as opa has warned us.