Skip to content

feat(extensions): B+C extensions framework with NIST 800-53 compliance extension#239

Open
ClintEastman02 wants to merge 4 commits into
awslabs:mainfrom
harmjeff:feat/clintae-community-extensions-clean
Open

feat(extensions): B+C extensions framework with NIST 800-53 compliance extension#239
ClintEastman02 wants to merge 4 commits into
awslabs:mainfrom
harmjeff:feat/clintae-community-extensions-clean

Conversation

@ClintEastman02
Copy link
Copy Markdown

@ClintEastman02 ClintEastman02 commented May 4, 2026

Summary

Implements the hybrid B+C extensions model proposed in #225:

  • Proposal B: NIST 800-53 ships as official compliance extension alongside security-baseline and property-based-testing
  • Proposal C: Empty community-extensions-index.yaml establishes the decentralized extension contract for future community contributions
  • Selection UX: Consolidated menu replaces per-extension opt-in prompts — scales to N extensions with a single question

What Changed

File Change
inception/requirements-analysis.md Step 5.1: per-extension opt-in -> consolidated selection menu
extensions/README.md Combined with NIST README, lists all available extensions
extensions/compliance/nist-800-53/ New official NIST 800-53 extension (opt-in + controls)
community-extensions-index.yaml Empty index template at repo root for Proposal C
core-workflow.md Added .aidlc/ path for AI-assisted setup

NIST 800-53 Extension

~90 control mappings across 3 NIST families mapped to specific AWS mechanisms:

Family Controls AWS Mechanisms
AC (Access Control) AC-3, AC-4, AC-6, AC-12, AC-17, AC-22 + enhancements SCPs, Declarative Policies, Config Rules, Security Hub
AU (Audit and Accountability) AU-2, AU-3, AU-5(2), AU-6, AU-12 CF Hooks, Config Rules, Security Hub
CA (Assessment and Monitoring) CA-3, CA-7 CF Hooks, Config Rules (GuardDuty)

The model selectively applies controls relevant to each project — a network isolation project triggers AC-4/AC-17; a logging project triggers AU-2/AU-12.

Extension Selection Menu

Instead of N sequential yes/no prompts, all opt-in extensions are presented in one table:

| # | Extension          | Category   | Description                       |
|---|--------------------|------------|-----------------------------------|
| 1 | Security Baseline  | security   | OWASP-mapped security rules       |
| 2 | NIST 800-53        | compliance | NIST 800-53 controls mapped to AWS|

Enter numbers to enable (comma-separated), "all", or "none".

Alignment with #225

#225 Consensus This PR
Scott: "Proposal B + Proposal C" Official extensions in extensions/, community via index
Jeff: "SHA-256 integrity pinning" Index schema includes sha256 per file, mandatory at reviewed
Jeff: "Maturity lifecycle" new -> incubating -> reviewed -> adopted -> official
Jeff: "No extension dependencies for v1" Not implemented — kept simple
Unified loader Same *.opt-in.md scanning handles both official and installed community extensions

Test plan

  • Run 4 test scenarios with NIST OFF — verify inception completes normally
  • Run 4 test scenarios with NIST ON — verify NIST controls appear in docs
  • Verify selection menu presents all extensions in one table
  • Verify disabled extensions don not load full rules files (context optimization)
  • Verify aidlc-state.md records extension enablement status
  • Verify stage compliance summaries include extension rule status

Full test results with metrics posted in PR comment.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

ClintEastman02 and others added 3 commits May 1, 2026 15:57
…compliance extension

Adds the extensions README and the first community extension for NIST 800-53 compliance controls, including opt-in manifest, control mappings, and documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds ~90 control mappings across Access Control (AC-3 through AC-22),
Audit and Accountability (AU-2 through AU-12), and Assessment/Monitoring
(CA-3, CA-7) families for context-load testing of the extension.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… menu

- Change extension opt-in from per-extension prompts to single selection menu
- Remove redundant NIST README (consolidated into extensions/README.md)
- Remove rule-manifest.yaml (not needed for official extensions)
- Add community-extensions-index.yaml template at repo root
- Align with awslabs#225 consensus: official (B) + community (C) hybrid model

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ClintEastman02
Copy link
Copy Markdown
Author

ClintEastman02 commented May 4, 2026

NIST Compliance Extension - Test Results Summary

8 test runs across 4 scenarios, comparing baseline (no-ext) vs compliance extension (with-ext).

Network Isolation

Metric No-Ext With-Ext Delta
Duration (s) 594 601 +7 (+1.2%)
Files generated 15 19 +4
Word count 11,763 14,447 +2,684 (+22.8%)
NIST references 6 56 +50 (+833%)
Unique control IDs 0 18 +18
SCP mentions 0 2 +2
GuardDuty mentions 0 5 +5
Config Rule mentions 0 0 --
Stages completed 4/4 4/4 --
AWS mechanism IDs 0 1 +1
Control IDs found (with-ext)

AC-3, AC-4, AC-6, AC-6(1), AC-6(10), AC-12, AC-17, AC-17(1), AC-17(3), AC-22, AU-2, AU-3, AU-5(2), AU-6, AU-6(1), AU-12, CA-3, CA-7

AWS mechanisms: AWS-GR_DISALLOW_VPN_CONNECTIONS


Audit Logging

Metric No-Ext With-Ext Delta
Duration (s) 1,342 568 -774 (-57.7%)
Files generated 15 15 --
Word count 12,565 10,856 -1,709 (-13.6%)
NIST references 15 29 +14 (+93%)
Unique control IDs 0 9 +9
SCP mentions 0 0 --
GuardDuty mentions 0 2 +2
Config Rule mentions 0 0 --
Stages completed 4/4 4/4 --
AWS mechanism IDs 0 0 --
Control IDs found (with-ext)

AC-3, AC-4, AC-6, AC-12, AU-2, AU-3, AU-6, AU-12, CA-7


Threat Detection

Metric No-Ext With-Ext Delta
Duration (s) 673 660 -13 (-1.9%)
Files generated 15 15 --
Word count 13,216 13,722 +506 (+3.8%)
NIST references 12 26 +14 (+117%)
Unique control IDs 0 21 +21
SCP mentions 0 1 +1
GuardDuty mentions 31 48 +17 (+55%)
Config Rule mentions 0 0 --
Stages completed 4/4 4/4 --
AWS mechanism IDs 0 0 --
Control IDs found (with-ext)

AC-3, AC-4, AC-6, AC-6(1), AC-6(9), AC-6(10), AC-12, AC-17, AC-22, AU-2, AU-3, AU-5(2), AU-6, AU-6(1), AU-12, CA-01, CA-02, CA-03, CA-04, CA-3, CA-7


Tenant Isolation

Metric No-Ext With-Ext Delta
Duration (s) 1,004 719 -285 (-28.4%)
Files generated 19 19 --
Word count 17,164 14,889 -2,275 (-13.3%)
NIST references 51 30 -21 (-41.2%)
Unique control IDs 11 23 +12 (+109%)
SCP mentions 0 0 --
GuardDuty mentions 0 12 +12
Config Rule mentions 0 0 --
Stages completed 4/4 4/4 --
AWS mechanism IDs 0 0 --
Control IDs found

No-ext: AC-01 .. AC-05, AU-01 .. AU-06 (11 unique)

With-ext: AC-1, AC-2, AC-3, AC-4, AC-5, AC-6, AC-12, AC-17, AU-2, AU-3, AU-6, AU-6(1), AU-12, CA-3, CA-7, SC-1, SC-2, SC-3, SC-4, SC-5, SC-6, SC-7, SC-8 (23 unique)


Aggregate Summary

Metric No-Ext (avg) With-Ext (avg) Delta
Duration (s) 903 637 -266 (-29.5%)
Files generated 16.0 17.0 +1.0
Word count 13,677 13,479 -198 (-1.4%)
NIST references 21.0 35.3 +14.3 (+68%)
Unique control IDs 2.8 17.8 +15.0 (+543%)
GuardDuty mentions 7.8 16.8 +9.0 (+116%)
Stages completed 4/4 4/4 --

Key Takeaways

  1. Control ID coverage is the headline metric: The extension increased unique NIST control IDs from an average of 2.8 to 17.8 per run -- a 5.4x improvement. Three of four no-ext runs produced zero control IDs.
  2. No performance penalty: With-ext runs averaged 29.5% faster (637s vs 903s), though this is likely variance rather than a causal effect.
  3. NIST reference density improved across 3 of 4 scenarios: The tenant-isolation case is the exception, where the no-ext run happened to produce more raw NIST mentions (51 vs 30) but with far fewer structured control IDs (11 vs 23).
  4. GuardDuty enrichment works: Threat-detection-with-ext had 55% more GuardDuty references; tenant-isolation went from 0 to 12.
  5. SCP and Config Rule coverage is still thin: Only 3 total SCP mentions across all with-ext runs, and zero Config Rule / conformance pack mentions anywhere. This is an area for extension improvement.
  6. All runs completed 4/4 stages -- the extension does not disrupt the AIDLC workflow.
  7. AWS mechanism IDs are rare: Only one (AWS-GR_DISALLOW_VPN_CONNECTIONS) was found across all 8 runs. The regex patterns or extension guidance may need tuning for CT/SH/CONFIG prefixed IDs.

Copy link
Copy Markdown
Member

@scottschreckengaust scottschreckengaust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks viable


## NIST 800-53 Compliance Controls

**Version**: 0.2.0
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does the 0.2.0 come from?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this came from the different versions I was going through when the different versions i can revert to 0.1.0

Copy link
Copy Markdown
Member

@scottschreckengaust scottschreckengaust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks viable, see comments for follow-ups


## Available Extensions

| Extension | Category | Description |
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an easy way to add the Level for the Maturity Lifecycle concept ?

Should NIST 800-53 compliance controls be enforced for this project? This extension maps NIST 800-53 controls to AWS Control Tower preventive and detective controls.

A) Yes — enforce all applicable NIST 800-53 / AWS Control Tower controls as blocking constraints (recommended for regulated AWS workloads)
B) Yes — but only advisory (SHOULD/MAY, non-blocking)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would a pick list of numbers (when presented a list of available extensions) select only the advisory choice?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now the users can write the answers as 1,3 or 1,2 and that enables the extensions they want.

… menu

- Update Extensions section in README to describe selection menu, add NIST
  extension to tree, add community extensions paragraph
- Update WORKING-WITH-AIDLC advanced tip to describe menu instead of
  per-extension opt-in
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label May 4, 2026
@ClintEastman02
Copy link
Copy Markdown
Author

How the Extension Affects Generated Documents

Side-by-side excerpts from the Network Isolation test — same prompt, same security baseline, only difference is NIST ON vs OFF.

Without NIST Extension

Generic security NFRs with no compliance traceability:

### NFR-03: Security — Administrative Access
- Remote administrative access MUST be exclusively through AWS Systems Manager Session Manager

With NIST Extension

A dedicated compliance section maps each requirement to a specific NIST control:

## 5. NIST 800-53 Compliance Requirements

| Control              | Implementation                                              |
|----------------------|-------------------------------------------------------------|
| AC-4 (Info Flow)     | VPC SGs + NACLs enforce traffic; no 0.0.0.0/0; VPC endpoints |
| AC-17 (Remote Access)| No VPN/SSH; SSM Session Manager only                        |
| AU-2 (Event Logging) | API execution logging; CloudWatch; X-Ray tracing            |
| CA-7 (Monitoring)    | GuardDuty with ECS runtime + malware protection             |

This mapping then carries into Application Design:

### NIST 800-53 Design Alignment
| AC-4 (Info Flow)     | VPC security groups + NACLs restrict all traffic flows |
| AC-17 (Remote Access)| SSM Session Manager only; no SSH/VPN                  |
| CA-7 (Monitoring)    | GuardDuty + Security Hub integration                  |

What This Means

Without the extension — reasonable security requirements but no framework traceability. With the extension — every security decision maps to a NIST control ID and that mapping carries from requirements through design, giving auditors a clear trail from framework to implementation.

@ai-ram-ramani
Copy link
Copy Markdown
Contributor

can you explain what test harness are you using.

@ai-ram-ramani
Copy link
Copy Markdown
Contributor

ai-ram-ramani commented May 5, 2026

Also when u say its an official NIST 800-53 compliance extension. Does this mean that if I use this extension my NIST 800-53 audits are covered in my application built using AI-DLC.

@ai-ram-ramani
Copy link
Copy Markdown
Contributor

What does this statement mean - produced zero control ID's : Control ID coverage is the headline metric: The extension increased unique NIST control IDs from an average of 2.8 to 17.8 per run -- a 5.4x improvement. Three of four no-ext runs produced zero control IDs..

@ai-ram-ramani
Copy link
Copy Markdown
Contributor

Why is this extension specific to control tower ?

@ai-ram-ramani
Copy link
Copy Markdown
Contributor

Compliance extensions have different trust requirements than quality/testing extensions. Suggest scoping them out of the public index and supporting them through the private extensions mechanism instead.

@scottschreckengaust
Copy link
Copy Markdown
Member

Compliance extensions are best suited in the "community" category.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge Halts a pull request from merging documentation Improvements or additions to documentation rules

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants