feature: Documentation command#1009
Conversation
92993db to
f9f3d0e
Compare
3d36716 to
73114d4
Compare
boranx
left a comment
There was a problem hiding this comment.
hey @xNok
Thanks for bringing this together, I actually very liked the idea of using go templates and embedding both so if the user has their own, overriding it to balance flexibility with functionality.
This can help us collectively gather feedback on how users actually use it and refactor to get it closer to the user needs.
+1 from me on implementation.
Just left some comments at first glance, will look at more.
191ca6e to
5b32548
Compare
7b3dbe6 to
61b27da
Compare
jalseth
left a comment
There was a problem hiding this comment.
Thanks for the PR! It is very thorough and including documentation on how to use the feature is much appreciated.
|
@jalseth, I have addressed your comment. It seems there is a broken test, and it fails on |
|
@xNok Please rebase the branch, the CI should now be passing |
530a4c1 to
0cb4f3e
Compare
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> refactor: change the way we access annotations I am testing different way to generate the doc. Having the compiler object is much easier than the annotation set only Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> fix: make title coherent in submodules documentation Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: ignore golden files in git Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: document and test the custom template feature Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: linting Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
…9.0 (open-policy-agent#1010) Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.68.0 to 0.69.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v0.68.0...v0.69.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
build(deps): bump github.com/open-policy-agent/opa from 0.68.0 to 0.69.0 (open-policy-agent#1010) Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.68.0 to 0.69.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v0.68.0...v0.69.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: go mod tidy
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
jalseth Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
0cb4f3e to
0396ea4
Compare
the name of the variable has changed to be more meaningful Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg>
|
fwiw, this feature looks similar to https://github.com/plexsystems/konstraint/blob/main/docs/cli/konstraint_doc.md maybe it makes sense to integrate the feature from konstraint? |
Kubetmetes are not the only thing you can validate with conftest, we use it for terraform and Backstage manifest, in which cases it makes no sense to use Konstraint |
|
This is neat! Thanks for working on this. Let's have it merged! |
|
Finally getting this over the finish line—appreciate everyone's input! |
* feat: parse the annotations Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> refactor: change the way we access annotations I am testing different way to generate the doc. Having the compiler object is much easier than the annotation set only Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * feat: generate section that can be used to generate documentations Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * feat: generate the documentation file Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * feat: add the command doc to conftest Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> fix: make title coherent in submodules documentation Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: improve test case Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: ignore golden files in git Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: document and test the custom template feature Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: linting Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * build(deps): bump github.com/open-policy-agent/opa from 0.68.0 to 0.69.0 (open-policy-agent#1010) Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.68.0 to 0.69.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v0.68.0...v0.69.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: Fixed changes requested by @boranx build(deps): bump github.com/open-policy-agent/opa from 0.68.0 to 0.69.0 (open-policy-agent#1010) Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.68.0 to 0.69.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v0.68.0...v0.69.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: go mod tidy * test: refactor to use Document instead of []Section Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: improve doc string in code Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: address changes requested by @ jalseth Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: remove error handling on file close Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * fix: revert bad renaming Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * fix: go mod tidy Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * fix: update template in acceptance test the name of the variable has changed to be more meaningful Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> --------- Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps): bump github.com/moby/buildkit from 0.18.0 to 0.18.1 (open-policy-agent#1024) * build(deps): bump golang from 1.23.3-alpine to 1.23.4-alpine (open-policy-agent#1025) * build(deps): bump alpine from 3.20.3 to 3.21.0 (open-policy-agent#1026) * build(deps): bump github.com/magiconair/properties from 1.8.7 to 1.8.9 (open-policy-agent#1027) Bumps [github.com/magiconair/properties](https://github.com/magiconair/properties) from 1.8.7 to 1.8.9. - [Release notes](https://github.com/magiconair/properties/releases) - [Commits](magiconair/properties@v1.8.7...v1.8.9) --- updated-dependencies: - dependency-name: github.com/magiconair/properties dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/CycloneDX/cyclonedx-go from 0.9.1 to 0.9.2 (open-policy-agent#1028) Bumps [github.com/CycloneDX/cyclonedx-go](https://github.com/CycloneDX/cyclonedx-go) from 0.9.1 to 0.9.2. - [Release notes](https://github.com/CycloneDX/cyclonedx-go/releases) - [Changelog](https://github.com/CycloneDX/cyclonedx-go/blob/master/.goreleaser.yml) - [Commits](CycloneDX/cyclonedx-go@v0.9.1...v0.9.2) --- updated-dependencies: - dependency-name: github.com/CycloneDX/cyclonedx-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: set jsonnet VM stack limits and add test coverage - Set MaxStack to 500 frames to prevent deep recursion - Set MaxStackTraceSize to 20 for cleaner errors - Add test for stack overflow prevention - Improve test structure with table-driven pattern Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * fix: improve handling for YAML version directives The YAML parser was incorrectly splitting documents with version directives (e.g. '%YAML 1.1') by treating the first '---' marker as a document separator. This caused the version directive to be separated from its document, resulting in parsing errors. Fixed document separation logic to keep version directive with its document. Added test cases for various YAML scenarios and improved test coverage. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * fix: add output and tests for GitHub and Azure DevOps Added missing test cases for GitHub and AzureDevOps output formats in output_test.go. Also added OutputAzureDevOps to the list of available output formats in Outputs(). Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * fix: remove redundant error check in push command Remove an unnecessary error check for configDesc creation in the push command. The NewDescriptorFromBytes function does not return an error, so checking for one was redundant and could never be triggered. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * build(deps): bump golang from 1.23.4-alpine to 1.23.5-alpine Bumps golang from 1.23.4-alpine to 1.23.5-alpine. --- updated-dependencies: - dependency-name: golang dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * fix: prevent policy file overwrite on downloads (open-policy-agent#1039) File existence check before downloading policies. Errors out and no overwrites. Maintains data integrity by preventing accidental policy overwrites. Added a test which verified the behaviour. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * fix: max stack size already set by jsonnet.MakeVM() Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * build(deps): bump cuelang.org/go from 0.11.0 to 0.12.0 Bumps cuelang.org/go from 0.11.0 to 0.12.0. --- updated-dependencies: - dependency-name: cuelang.org/go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump github.com/moby/buildkit from 0.18.1 to 0.19.0 Bumps [github.com/moby/buildkit](https://github.com/moby/buildkit) from 0.18.1 to 0.19.0. - [Release notes](https://github.com/moby/buildkit/releases) - [Commits](moby/buildkit@v0.18.1...v0.19.0) --- updated-dependencies: - dependency-name: github.com/moby/buildkit dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump alpine from 3.21.0 to 3.21.2 Bumps alpine from 3.21.0 to 3.21.2. --- updated-dependencies: - dependency-name: alpine dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump github.com/hashicorp/go-getter from 1.7.6 to 1.7.8 Bumps [github.com/hashicorp/go-getter](https://github.com/hashicorp/go-getter) from 1.7.6 to 1.7.8. - [Release notes](https://github.com/hashicorp/go-getter/releases) - [Changelog](https://github.com/hashicorp/go-getter/blob/main/.goreleaser.yml) - [Commits](hashicorp/go-getter@v1.7.6...v1.7.8) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-getter dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: optimize yaml document separator handling Based on PR feedback: - use byte slices instead of string concat - set common byte sequences as vars Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * feat: enable relative jsonnet imports by setting a path-aware importer - Introduce a new `PathAwareParser` interface that extends `Parser` with `SetPath(path string)`. - Update `parseConfigurations` to detect if a parser implements `PathAwareParser`, and call `SetPath(path)`. - Implement `SetPath` in the Jsonnet parser, setting the JPath to the file’s directory. - Add tests to ensure that relative imports in Jsonnet now work as expected. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * fix: correct linters-settings in .golangci.yaml to enable misspell * Fix typo in `.golangci.yaml` to enable linter settings * Fix marshal/unmarshal spelling across codebase Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * chore: add nilness check to govet linter * Add nilness check to govet linter configuration See open-policy-agent#1041 for an example Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * test(registry): add test for credentials store init failure Add a test case that verifies SetupClient returns an error when the Docker config file is not readable due to permissions. This ensures the credentials store initialization failure is properly handled. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * feature: Documentation command (open-policy-agent#1009) * feat: parse the annotations Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> refactor: change the way we access annotations I am testing different way to generate the doc. Having the compiler object is much easier than the annotation set only Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * feat: generate section that can be used to generate documentations Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * feat: generate the documentation file Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * feat: add the command doc to conftest Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> fix: make title coherent in submodules documentation Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: improve test case Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: ignore golden files in git Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: document and test the custom template feature Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: linting Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * build(deps): bump github.com/open-policy-agent/opa from 0.68.0 to 0.69.0 (open-policy-agent#1010) Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.68.0 to 0.69.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v0.68.0...v0.69.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: Fixed changes requested by @boranx build(deps): bump github.com/open-policy-agent/opa from 0.68.0 to 0.69.0 (open-policy-agent#1010) Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.68.0 to 0.69.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v0.68.0...v0.69.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> chore: go mod tidy * test: refactor to use Document instead of []Section Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: improve doc string in code Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: address changes requested by @ jalseth Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * chore: remove error handling on file close Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * fix: revert bad renaming Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * fix: go mod tidy Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> * fix: update template in acceptance test the name of the variable has changed to be more meaningful Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> --------- Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * test(policy): improve engine test coverage (open-policy-agent#1055) * test(policy): improve engine test coverage Add TestLoadWithData and TestNamespaces to increase test coverage for the policy engine. Tests a few edge cases such as duplicate namespaces. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * refactor(policy): replace memfs with testing/fstest Migrate tests to use Go's standard library fstest.MapFS instead of custom memfs implementation. - Update TestProblematicIf and TestNamespaces to use fstest.MapFS - Remove memfs import and dependency - Convert test file content handling to use MapFile structs - Maintain existing test functionality while using official FS impl Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * docs: explain fstest.MapFS conversion in tests The type conversion from map[string]*fstest.MapFile to fstest.MapFS triggers an 'unnecessary conversion' linter warning. Add explanatory comment to document why this conversion is required - it ensures proper fs.FS interface implementation needed by loader.WithFS. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> --------- Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * build(deps): bump github.com/open-policy-agent/opa from 0.70.0 to 1.1.0 (open-policy-agent#1050) * build(deps): bump github.com/open-policy-agent/opa from 0.70.0 to 1.1.0 Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.70.0 to 1.1.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v0.70.0...v1.1.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * ci: Disable deprecated code check for golangci-lint This is currently blocking the upgrade to the OPA v1 package, and as a general rule I don't think hard blocking updates and releases on use of deprecated but still working code is good. Signed-off-by: James Alseth <james@jalseth.me> * document: Use V1 AST The V0 path is currently missing the BuildAnnotationSet function which breaks the build. This will be fixed, but the V0 path is an alias to the V1 path, so this is effectively the same. This does not have an impact on the required Rego for conftest users. Signed-off-by: James Alseth <james@jalseth.me> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: James Alseth <james@jalseth.me> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: James Alseth <james@jalseth.me> * build(deps): bump golang from 1.23.5-alpine to 1.23.6-alpine (open-policy-agent#1062) Bumps golang from 1.23.5-alpine to 1.23.6-alpine. --- updated-dependencies: - dependency-name: golang dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat(engine): add query metadata to evaluation results (open-policy-agent#1061) * feat(engine): add query metadata to evaluation results Add query information to result metadata to enable output formatters to utilize the originating query in their output. This helps track which policy rule triggered each result. The change: - Adds query metadata to both string and map return types - Ensures metadata map is properly initialized - Adds comprehensive tests covering single and multiple results Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * test: split checks in TestQueryMetadata Check msg and meta in separate blocks, with separate err Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> --------- Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * engine: Refactor to allow for Rego version to be specified (open-policy-agent#1059) Signed-off-by: James Alseth <james@jalseth.me> * feat(parser): handle UTF-8 BOM in JSON input (open-policy-agent#1065) Strip UTF-8 BOM (byte order mark) from JSON input before parsing, allowing users to directly use files containing BOM with conftest. Matches OPA's implementation. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * test(plugin): add comprehensive plugin package tests (open-policy-agent#1056) * test(plugin): add comprehensive plugin package tests Add tests for Load, FindAll and plugin Exec functionality. Includes test coverage for plugin loading, discovery and command execution. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * fix: proper file paths for plugin tests CacheDirectory() will resolve to current working directory if pre-defined directory structure is missing. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> --------- Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * feat: Implement SARIF output (open-policy-agent#1042) * feat: add SARIF output format support Add Static Analysis Results Interchange Format (SARIF) v2.1.0 output support to conftest. SARIF is a standard JSON format for static analysis tools. - SARIF v2.1.0 schema compliance - Includes file locations and rule metadata - Tracks execution timing and status - Test coverage - Documentation Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * feat(output): implement SARIF output using go-sarif library Add Static Analysis Results Interchange Format (SARIF) v2.1.0 output support using the go-sarif library. This provides a standard JSON format for static analysis results with proper schema compliance. Key changes: - Use go-sarif/v2 library instead of custom implementation - Support all result types (failures, warnings, exceptions, successes) - Add comprehensive test coverage with JSON comparison - Document new output format in options.md The SARIF output includes: - File locations and rule metadata - Proper result levels (error/warning/note/none) - Execution status and exit codes - Rule properties from result metadata Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * refactor: address pr comments - refactor: remove getRuleIndex Use direct map lookups instead - refactor: succinct map lookups Map lookup with a fallback - refactor: move result type logic to addResult func Cleaner code, while not really idiomatic due to go-sarif library design. - fix: treat exceptions as success A file with only exceptions will be treated as a success. Exceptions will still be logged (with level "note") for visibility. The exit code will be 0 (success) when there are only exceptions. - refactor: simplify hasFailures and hasWarnings Risk of typo is too high - refactor: treat exceptions as successes in SARIF output Exceptions are now treated as successes in the SARIF output, removing the separate exception handling. - test: type safe test input for SARIF Probably helps writing further test cases, instead of bare JSON - refactor: use google/go-cmp for json diff Based on PR comment Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> --------- Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * chore: Change empty interface{} to any throughout the codebase (open-policy-agent#1057) The any type is a clearer intent than an empty interface. Signed-off-by: James Alseth <james@jalseth.me> * chore: Update all examples to OPA V1 syntax (open-policy-agent#1058) Signed-off-by: James Alseth <james@jalseth.me> * refactor(output): Add CheckResults type and helpers (open-policy-agent#1063) Adding a new type for a slice of CheckResult to add helper methods for checking if there were failures, warnings, or exceptions. Signed-off-by: James Alseth <james@jalseth.me> * feat(test): Enable inter-query cache (open-policy-agent#1073) This improves performance for some Rego policies, such as those that use the http.send builtin. This is only enabled for the "conftest test" comamnd to avoid introducing flaky unit tests with "conftest verify" due to the caching. Signed-off-by: James Alseth <james@jalseth.me> * build(deps): bump github.com/open-policy-agent/opa from 1.1.0 to 1.2.0 (open-policy-agent#1080) * build(deps): bump github.com/open-policy-agent/opa from 1.1.0 to 1.2.0 Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 1.1.0 to 1.2.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v1.1.0...v1.2.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * build: Run `go mod tidy`. Signed-off-by: James Alseth <james@jalseth.me> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: James Alseth <james@jalseth.me> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: James Alseth <james@jalseth.me> * build(deps): bump alpine from 3.21.2 to 3.21.3 (open-policy-agent#1072) Bumps alpine from 3.21.2 to 3.21.3. --- updated-dependencies: - dependency-name: alpine dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/moby/buildkit from 0.19.0 to 0.20.0 (open-policy-agent#1076) Bumps [github.com/moby/buildkit](https://github.com/moby/buildkit) from 0.19.0 to 0.20.0. - [Release notes](https://github.com/moby/buildkit/releases) - [Commits](moby/buildkit@v0.19.0...v0.20.0) --- updated-dependencies: - dependency-name: github.com/moby/buildkit dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * refactor(ci): replace Makefile-based Docker builds with GitHub Action… (open-policy-agent#1075) * refactor(ci): replace Makefile-based Docker builds with GitHub Actions for easier maintenance Signed-off-by: Jesse Amamgbu <amamgbujesse@yahoo.com> * build(deps): bump golang from 1.23.6-alpine to 1.24.0-alpine (open-policy-agent#1071) * build(deps): bump golang from 1.23.6-alpine to 1.24.0-alpine Bumps golang from 1.23.6-alpine to 1.24.0-alpine. --- updated-dependencies: - dependency-name: golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * build: Bump Go to 1.24. Signed-off-by: James Alseth <james@jalseth.me> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: James Alseth <james@jalseth.me> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: James Alseth <james@jalseth.me> * feat: add --absolute-paths flag to pull command (open-policy-agent#1078) Add a new flag to the pull command that allows preserving absolute paths when downloading policies. By default, absolute paths in the --policy flag are treated as relative paths from the current directory. With the new --absolute-paths flag, users can specify absolute paths to download policies to. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * docs: clarify --trace flag behavior with --output flag (open-policy-agent#1060) Improve documentation to make it clear that when both --trace and --output flags are specified, the output format takes priority over tracing. This helps users understand that they cannot combine tracing with custom output formats like JSON or table. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * fix: Conftest can now successfully load files using a file URL (e.g., `file:///C:/path/to/data.yaml`) on windows (open-policy-agent#999) * fix: Conftest encounters errors on Windows when loading file paths that include drive letters (e.g., `C:/path/to/data.yaml`). Even when using a file URL (e.g., `file:///C:/path/to/data.yaml`), we still face issues. With these code changes, Conftest can now successfully load files using a file URL (e.g., `file:///C:/path/to/data.yaml`). We opted for file URLs instead of paths with drive letters (e.g., `C:/path/to/data.yaml`) because OPA does not support file paths with drive letters. For more details, see [this issue comment](open-policy-agent/opa#6922 (comment)). Resolves: open-policy-agent#979 Signed-off-by: Punith C K <pck@veracode.com> * fix: Removing WithProcessAnnotation(true) which is not needed for loading data files Signed-off-by: Punith C K <punithck@itsupports-Laptop.local> * fix: Conftest can now successfully load files using a file URL (e.g., file:///C:/path/to/data.yaml) on windows Removing duplicate code Signed-off-by: Punith C K <pck@veracode.com> --------- Signed-off-by: Punith C K <punithck@itsupports-Laptop.local> Signed-off-by: Punith C K <pck@veracode.com> Co-authored-by: Punith C K <punithck@itsupports-Laptop.local> * ci: Revert "replace Makefile-based Docker builds with GitHub Action… (open-policy-agent#1075)" (open-policy-agent#1081) This reverts commit 854183b. Signed-off-by: James Alseth <james@jalseth.me> * build(deps): bump github.com/moby/buildkit from 0.20.0 to 0.20.1 (open-policy-agent#1083) Bumps [github.com/moby/buildkit](https://github.com/moby/buildkit) from 0.20.0 to 0.20.1. - [Release notes](https://github.com/moby/buildkit/releases) - [Commits](moby/buildkit@v0.20.0...v0.20.1) --- updated-dependencies: - dependency-name: github.com/moby/buildkit dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: add documentation feature to the navigation bar (open-policy-agent#1087) Signed-off-by: boranx <boran.seref@gmail.com> * feat: add pre-commit hook support (open-policy-agent#1077) * feat: add pre-commit hook support Add pre-commit integration to validate configurations against OPA policies: - Create .pre-commit-hooks.yaml with conftest test/verify hook configs - Add pre-commit hook tests using bats - Install pre-commit in CI workflow Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * chore: add teardown to pre-commit test Add teardown function so we clean up the test config. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * chore: remove verbose from pre-commit test Test checks the return code only Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * chore: suggestions from the pr Co-authored-by: Boran Seref <boran.seref@gmail.com> Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * chore: add Makefile step for installing deps Since acceptance tests require pre-commit lets make sure it's available - if not, install with pip. This is bundled with the 'make test-acceptance' step so that local and CI environments behave the same way. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * chore: use isolated temp repo for testing pre-commit - Create a temporary Git repository for each test run - Configure Git with test-specific user info and disabled signing - Use absolute paths for pre-commit hook configuration - Clean up test artifacts properly after each run - Remove dependency on main repository state Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> --------- Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * build(deps): bump golang from 1.24.0-alpine to 1.24.1-alpine (open-policy-agent#1086) Bumps golang from 1.24.0-alpine to 1.24.1-alpine. --- updated-dependencies: - dependency-name: golang dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/BurntSushi/toml from 1.4.0 to 1.5.0 (open-policy-agent#1089) Bumps [github.com/BurntSushi/toml](https://github.com/BurntSushi/toml) from 1.4.0 to 1.5.0. - [Release notes](https://github.com/BurntSushi/toml/releases) - [Commits](BurntSushi/toml@v1.4.0...v1.5.0) --- updated-dependencies: - dependency-name: github.com/BurntSushi/toml dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ci: Remove PR workflow access to all permissions from GITHUB_TOKEN (open-policy-agent#1088) Signed-off-by: James Alseth <james@jalseth.me> * build(deps): bump github.com/moby/buildkit from 0.20.1 to 0.20.2 (open-policy-agent#1091) Bumps [github.com/moby/buildkit](https://github.com/moby/buildkit) from 0.20.1 to 0.20.2. - [Release notes](https://github.com/moby/buildkit/releases) - [Commits](moby/buildkit@v0.20.1...v0.20.2) --- updated-dependencies: - dependency-name: github.com/moby/buildkit dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/open-policy-agent/opa from 1.2.0 to 1.3.0 (open-policy-agent#1092) Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 1.2.0 to 1.3.0. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v1.2.0...v1.3.0) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump golang from 1.24.1-alpine to 1.24.2-alpine (open-policy-agent#1096) Bumps golang from 1.24.1-alpine to 1.24.2-alpine. --- updated-dependencies: - dependency-name: golang dependency-version: 1.24.2-alpine dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump cuelang.org/go from 0.12.0 to 0.12.1 (open-policy-agent#1094) Bumps cuelang.org/go from 0.12.0 to 0.12.1. --- updated-dependencies: - dependency-name: cuelang.org/go dependency-version: 0.12.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/magiconair/properties from 1.8.9 to 1.8.10 (open-policy-agent#1097) Bumps [github.com/magiconair/properties](https://github.com/magiconair/properties) from 1.8.9 to 1.8.10. - [Release notes](https://github.com/magiconair/properties/releases) - [Commits](magiconair/properties@v1.8.9...v1.8.10) --- updated-dependencies: - dependency-name: github.com/magiconair/properties dependency-version: 1.8.10 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * deps: Bump hcl2json to v0.6.7 (open-policy-agent#1074) Signed-off-by: James Alseth <james@jalseth.me> * refactor(ci): replace Makefile-based Docker builds with GitHub Action (open-policy-agent#1082) Signed-off-by: Jesse Amamgbu <amamgbujesse@yahoo.com> * docs: update default template link to use absolute URL (open-policy-agent#1099) Change the relative link to the default template to use an absolute URL pointing to the raw GitHub content for better accessibility. Currently the link is broken since the file from the relative path is not deployed to the docs site. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * chore: Update Github Actions via Dependabot (open-policy-agent#1100) Signed-off-by: Manuel Rüger <manuel@rueg.eu> * build(deps): bump golangci/golangci-lint-action from 6 to 7 (open-policy-agent#1103) * build(deps): bump golangci/golangci-lint-action from 6 to 7 Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 6 to 7. - [Release notes](https://github.com/golangci/golangci-lint-action/releases) - [Commits](golangci/golangci-lint-action@v6...v7) --- updated-dependencies: - dependency-name: golangci/golangci-lint-action dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * ci: Run "golangci-lint migrate" Signed-off-by: James Alseth <james@jalseth.me> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: James Alseth <james@jalseth.me> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: James Alseth <james@jalseth.me> * build(deps): bump actions/setup-go from 4 to 5 (open-policy-agent#1102) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to 5. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump bats-core/bats-action from 1.5.4 to 3.0.1 (open-policy-agent#1104) Bumps [bats-core/bats-action](https://github.com/bats-core/bats-action) from 1.5.4 to 3.0.1. - [Release notes](https://github.com/bats-core/bats-action/releases) - [Commits](bats-core/bats-action@1.5.4...3.0.1) --- updated-dependencies: - dependency-name: bats-core/bats-action dependency-version: 3.0.1 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ci: Move docker build to separate job in the PR workflow (open-policy-agent#1105) This step is slow to run and only needs to be tested after everything else. Signed-off-by: James Alseth <james@jalseth.me> * feat(runner): add support for symlinks (open-policy-agent#1098) Signed-off-by: Dieter Bocklandt <dieter.bocklandt@embark-studios.com> * feat(output): redirect trace output to stderr (open-policy-agent#1084) Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> * build(deps): bump github.com/moby/buildkit from 0.20.2 to 0.21.0 (open-policy-agent#1101) Bumps [github.com/moby/buildkit](https://github.com/moby/buildkit) from 0.20.2 to 0.21.0. - [Release notes](https://github.com/moby/buildkit/releases) - [Commits](moby/buildkit@v0.20.2...v0.21.0) --- updated-dependencies: - dependency-name: github.com/moby/buildkit dependency-version: 0.21.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/moby/buildkit from 0.21.0 to 0.21.1 (open-policy-agent#1111) Bumps [github.com/moby/buildkit](https://github.com/moby/buildkit) from 0.21.0 to 0.21.1. - [Release notes](https://github.com/moby/buildkit/releases) - [Commits](moby/buildkit@v0.21.0...v0.21.1) --- updated-dependencies: - dependency-name: github.com/moby/buildkit dependency-version: 0.21.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/open-policy-agent/opa from 1.3.0 to 1.4.1 (open-policy-agent#1113) * build(deps): bump github.com/open-policy-agent/opa from 1.3.0 to 1.4.1 Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 1.3.0 to 1.4.1. - [Release notes](https://github.com/open-policy-agent/opa/releases) - [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md) - [Commits](open-policy-agent/opa@v1.3.0...v1.4.1) --- updated-dependencies: - dependency-name: github.com/open-policy-agent/opa dependency-version: 1.4.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: go mod tidy Signed-off-by: James Alseth <james@jalseth.me> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: James Alseth <james@jalseth.me> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: James Alseth <james@jalseth.me> * docs: Make examples in the docs compatible with v1 syntax (open-policy-agent#1115) Signed-off-by: James Alseth <james@jalseth.me> * cli: Make Rego v1 syntax the default (open-policy-agent#1114) Signed-off-by: James Alseth <james@jalseth.me> * build(deps): bump golangci/golangci-lint-action from 7 to 8 (open-policy-agent#1119) Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 7 to 8. - [Release notes](https://github.com/golangci/golangci-lint-action/releases) - [Commits](golangci/golangci-lint-action@v7...v8) --- updated-dependencies: - dependency-name: golangci/golangci-lint-action dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/google/go-jsonnet from 0.20.0 to 0.21.0 (open-policy-agent#1120) Bumps [github.com/google/go-jsonnet](https://github.com/google/go-jsonnet) from 0.20.0 to 0.21.0. - [Release notes](https://github.com/google/go-jsonnet/releases) - [Changelog](https://github.com/google/go-jsonnet/blob/master/.goreleaser.yml) - [Commits](google/go-jsonnet@v0.20.0...v0.21.0) --- updated-dependencies: - dependency-name: github.com/google/go-jsonnet dependency-version: 0.21.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: Update README.md to make it compatible with v1 syntax (open-policy-agent#1122) Signed-off-by: William Burton <william17.burton@gmail.com> * fix: revert makefile changes --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Ville Vesilehto <ville@vesilehto.fi> Signed-off-by: Alexandre Couedelo <a.couedelo@efg.gg> Signed-off-by: James Alseth <james@jalseth.me> Signed-off-by: Jesse Amamgbu <amamgbujesse@yahoo.com> Signed-off-by: Punith C K <punithck@itsupports-Laptop.local> Signed-off-by: Punith C K <pck@veracode.com> Signed-off-by: boranx <boran.seref@gmail.com> Signed-off-by: Manuel Rüger <manuel@rueg.eu> Signed-off-by: Dieter Bocklandt <dieter.bocklandt@embark-studios.com> Signed-off-by: William Burton <william17.burton@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ville Vesilehto <ville@vesilehto.fi> Co-authored-by: Anders Eknert <anders@styra.com> Co-authored-by: Alexandre Couedelo <nokwebspace@gmail.com> Co-authored-by: James Alseth <james@jalseth.me> Co-authored-by: Jesse Amamgbu <amamgbujesse@yahoo.com> Co-authored-by: pckvcode <pck@veracode.com> Co-authored-by: Punith C K <punithck@itsupports-Laptop.local> Co-authored-by: Boran Seref <boran.seref@gmail.com> Co-authored-by: Manuel Rüger <manuel@rueg.eu> Co-authored-by: Dieter Bocklandt <dieterbocklandt@gmail.com> Co-authored-by: William Burton <william17.burton@gmail.com>


I found myself wanting to generate reference documentation for my policies, and I figured it would be nice if this was a built-in feature of conftest.
While the way of structuring the documentation can be opinionated, I think using the FlatAnnotationsRefSet is a safe bet, as OPA's core does all the heavy lifting of organizing annotations and rules.
My initial attempt was to combine
FlatAnnotationsRefSetand Go-template and offer the least opinionated approach to documentation format that can be easily overridden. However, I had some trouble properly formatting titles (H1, H2, H3, etc...) and had to introduce some custom logic to handle it. (As a side note this could potentially be handles by adding template function libraries such as sprig)