nist_sync: export NIST 800-53 Rev 5 controls to Gemara format #14797
Draft
ggbecker wants to merge 2 commits into
Draft
nist_sync: export NIST 800-53 Rev 5 controls to Gemara format #14797ggbecker wants to merge 2 commits into
ggbecker wants to merge 2 commits into
Conversation
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
Implements OPENSCAP-6801. Adds tooling to export ComplianceAsCode NIST 800-53 Rev 5 control files (rhel8/9/10) to three Gemara artifact types: GuidanceCatalog (utils/nist_sync/gemara/guidance.py) Abstract NIST 800-53 standard layer built from the OSCAL catalog at utils/nist_sync/data/nist_800_53_rev5_catalog.json. Includes official control statement prose (with parameter substitution) as objectives, OSCAL guidance prose as rationale, and baseline applicability (low, moderate, high) from the three baseline JSON files. ControlCatalog (utils/nist_sync/gemara/catalog.py) Per-product: maps 1,196 NIST controls to XCCDF rule IDs as assessment requirements. NIST families become groups; baselines become applicability- groups. CaC control status maps to Gemara lifecycle states. MappingDocument (utils/nist_sync/gemara/mapping.py) Per-product: bidirectional traceability between NIST control IDs and ComplianceAsCode rule IDs with relationship type, strength, and confidence level derived from the CaC control status. All three artifact types pass CUE schema validation against the Gemara 1.1.0 schema (cue vet). Supporting files: gemara/status_map.py - CaC status → Gemara state/relationship/confidence gemara/schema.py - schema constants and structural validators export_to_gemara.py - CLI: --products, --validate, --gemara-schema, --no-guidance test_gemara_export.py - 87-test suite: structure, cross-references, accuracy generate_complyctl_bundle.py - OCI bundle generator for complyctl testing test_complyctl_e2e.sh - E2E test in UBI9 container against RHEL9 data stream .github/workflows/export-to-gemara.yml - CI on NIST control/script changes
…ibility Three root causes were found by reading the complytime-providers source code and inspecting provider gRPC logs: 1. Assessment-plan id must equal requirement-id go-gemara v0.0.1 populates AssessmentConfiguration.RequirementID from the plan 'id' field, not 'requirement-id'. The 'ap-' prefix we added made the id (e.g. 'ap-accounts_tmout') not match any data stream rule. Fix: use the short CaC rule name as both id and requirement-id. 2. requirement-id must be the short CaC rule name (no XCCDF prefix) filterValidRules() strips 'xccdf_org.ssgproject.content_rule_' from data stream rule IDs before comparing. Passing full XCCDF IDs caused no match. 3. 'datastream' target variable bypasses OS auto-detection On UBI9, ID_LIKE=fedora causes the provider to pick ssg-fedora-ds.xml before ssg-rhel9-ds.xml (alphabetical filesystem walk). Setting the 'datastream' variable in complytime.yaml targets the correct product data stream directly without overriding /etc/os-release. Also updates test_complyctl_e2e.sh to reflect the correct setup: no /etc/os-release override needed, uses 'datastream' variable and 'cis' as the XCCDF tailoring base profile (covers all 22 NIST 800-53 moderate rules). Verified end-to-end in a UBI9 container against ssg-rhel9-ds.xml: complyctl get → bundle synced complyctl generate → tailoring.xml created (22 requirements) complyctl scan → 3 passed, 1 failed (configure_custom_crypto_policy_cis) Results: arf.xml + evaluation-log written to /results/
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Adds
utils/nist_sync/export_to_gemara.pyto export ComplianceAsCode NIST 800-53 Rev 5 control files (rhel8/9/10) to three Gemara artifact types per product: GuidanceCatalog (abstract NIST standard text from OSCAL), ControlCatalog (NIST controls → XCCDF rule IDs), and MappingDocument (NIST↔rule traceability). All artifacts pass CUE schema validation against Gemara 1.1.0.Adds
generate_complyctl_bundle.pyto package the output as a complyctl-compatible OCI bundle andtest_complyctl_e2e.shto runcomplyctl get/generate/scanagainst RHEL9 content in a UBI9 container.Adds
.github/workflows/export-to-gemara.ymlto run the export and CUE validation on CI whenever NIST control files or export tooling change.Restructures NIST 800-53 control files to nest enhancements (e.g.,
ac-7.1) under their base control (ac-7) across rhel8/9/10, and updates the HTML viewer to display the nested hierarchy.Rationale:
Review Hints:
Quick validation (no build needed):
The nested control structure change affects all family YAML files under
products/{rhel8,rhel9,rhel10}/controls/nist_800_53/— reviewac.ymlin any product to see the before/after format.