Skip to content

nist_sync: split ControlCatalogs by NIST family - #15115

Open
ggbecker wants to merge 1 commit into
ComplianceAsCode:masterfrom
ggbecker:split-gemara-format-into-groups
Open

nist_sync: split ControlCatalogs by NIST family#15115
ggbecker wants to merge 1 commit into
ComplianceAsCode:masterfrom
ggbecker:split-gemara-format-into-groups

Conversation

@ggbecker

Copy link
Copy Markdown
Member

Description:

  • Add split-by-family Gemara ControlCatalog generation for NIST SP 800-53.
  • Generate one catalog per NIST family for each product, matching the complytime-policies repository layout.
  • Update PR staging to copy the family catalogs into the expected governance catalog directories.
  • Disable unwanted YAML aliases during Gemara and bundle generation.

Rationale:

  • The split catalog format is required by complytime-policies and keeps each catalog scoped to a single NIST 800-53 family.
  • Smaller family-specific catalogs make updates, reviews, and downstream consumption easier.
  • Disabling automatic aliases prevents spurious YAML anchors caused by object ID reuse during serialization.

Review Hints:

  • Review the changes together across:

    • utils/nist_sync/export_to_gemara.py
    • utils/nist_sync/gemara/catalog.py
    • utils/nist_sync/generate_policies_pr.py
    • utils/nist_sync/generate_complyctl_bundle.py
  • Generate artifacts for all supported products:

    python3 utils/nist_sync/export_to_gemara.py \
        --products rhel8,rhel9,rhel10,fedora \
        --output-dir build/gemara \
        --validate

@ggbecker ggbecker added this to the 0.1.83 milestone Sep 10, 2026
@ggbecker ggbecker added the Infrastructure Our content build system label Sep 10, 2026
@ggbecker
ggbecker marked this pull request as draft September 10, 2026 19:13
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Used by openshift-ci bot. label Sep 10, 2026
…iases

Match the layout complytime-policies uses (see complytime/complytime-
policies#55): one ControlCatalog per NIST 800-53 family (AC, AU, SI, ...)
alongside the existing combined catalog, so files stay reviewable and
each catalog's 'groups' scope is a single family.

- gemara/catalog.py: add GemaraCatalogBuilder.build_by_group(), returning
  one catalog dict per family with a family-scoped metadata id/description
  and title suffix.
- export_to_gemara.py: write the per-family catalogs to
  build/gemara/{product}/by_group/nist-800-53-rev5-{product}-{family}-catalog.yaml.
- generate_policies_pr.py: stage those into
  governance/catalogs/nist-800-53/{product}/nist-800-53-{product}/, matching
  the complytime-policies directory structure.

Also fix a pre-existing bug hit while generating these: ruamel.yaml aliases
list/dict objects by id(), and CPython can reuse the same id() for distinct
short-lived lists of equal length (e.g. repeated ['fedora-low'] built fresh
in a loop). This silently collapsed unrelated 'applicability' lists into
YAML anchors/aliases (&id001 / *id001) in the generated catalogs. Set
representer.ignore_aliases = lambda data: True in every script that dumps
these structures (export_to_gemara.py, generate_policies_pr.py,
generate_complyctl_bundle.py) to disable aliasing entirely.
@ggbecker
ggbecker force-pushed the split-gemara-format-into-groups branch from b1a6a65 to 1f5094a Compare September 11, 2026 09:41
@ggbecker
ggbecker marked this pull request as ready for review September 11, 2026 09:45
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Used by openshift-ci bot. label Sep 11, 2026
@openshift-ci

openshift-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

@ggbecker: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/4.12-images 1f5094a link true /test 4.12-images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Infrastructure Our content build system

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant