Implement Institutional Memory Framework: Adaptive Learning System for AI Runbooks

rules_bank/adaptive_learning_framework.md

@@ -0,0 +1,79 @@
+
+
+# Feature Proposal: Adaptive Learning Framework for AI Runbooks
+
+This document reframes the original "Adaptive Learning" proposal into an actionable implementation plan that aligns with the existing `rules_bank` and `personas` structure.
+
+## Core Concept: From Static Runbooks to Dynamic Institutional Memory
+
+The goal is to create a system where the agent can learn from analyst feedback and operational experience to improve its execution of runbooks. This moves beyond static procedures to a dynamic system that adapts to our organization's unique security context. We will call this `Institutional Memory`.
+
+## Proposed Structure
+
+We will introduce a new directory: `rules_bank/memories/`. This directory will store the learned procedures and context.
+
+### Memory File Format
+
+Each file within `rules_bank/memories/` will be a Markdown file representing a single "memory." The filename will correspond to the runbook it modifies, with a persona-specific suffix.
+
+**Example Filename:** `triage_alerts_soc_analyst_tier_1.md`
+
+**File Content:**
+
+```markdown
+---
+runbook: "run_books/triage_alerts.md"
+persona: "personas/soc_analyst_tier_1.md"
+source_step: "Step 3: IOC Enrichment"
+confidence: 0.95
+last_updated: "2025-08-23"
+---
+
+## Analyst Feedback
+
+"For our organization, when triaging alerts from the 'XYZ' sensor, we must always perform a historical lookup in our internal 'ThreatDB' before proceeding with public enrichment. This is a critical step that is currently missing."
+
+## Original Procedure (for context)
+
+- **Tool:** `secops-mcp enrich_ioc`
+- **Parameters:** `ioc={ioc_value}`
+
+## Derived Procedure
+
+1.  **Internal Enrichment:**
+    - **Tool:** `internal_db query`
+    - **Parameters:** `database=ThreatDB, query={ioc_value}`
+2.  **Conditional Public Enrichment:**
+    - **Condition:** If internal enrichment returns no results.
+    - **Tool:** `secops-mcp enrich_ioc`
+    - **Parameters:** `ioc={ioc_value}`
+
+## Application Log
+
+- **2025-08-23:** Memory created based on feedback from analyst 'D. Anderson'.
+- **2025-08-24:** Procedure successfully applied to Alert #4512.
+```
+
+## Workflow
+
+1.  **Initiation:** An agent, operating under a specific persona (e.g., `soc_analyst_tier_1`), is tasked with executing a runbook (e.g., `triage_alerts.md`).
+
+2.  **Memory Check:** Before executing a step in the runbook, the agent checks the `rules_bank/memories/` directory for a corresponding memory file (e.g., `triage_alerts_soc_analyst_tier_1.md`).
+
+3.  **Procedure Proposal:** If a relevant, high-confidence memory is found, the agent presents the "Derived Procedure" to the analyst as the recommended course of action for that step.
+
+4.  **Analyst Interaction & Feedback:**
+    - The analyst can **approve** the derived procedure, and the agent executes it.
+    - The analyst can **reject** it and proceed with the original runbook step.
+    - The analyst can provide **new feedback** in natural language (e.g., "That's close, but for this specific alert type, also check the 'LogRepo' system.").
+
+5.  **Memory Formation/Update:**
+    - New feedback is processed by a meta-agent.
+    - A new memory file is created, or an existing one is updated. The `confidence` score may be adjusted based on the feedback. The `Application Log` is updated.
+
+## Benefits
+
+*   **Context-Aware Execution:** Aligns agent behavior with our organization's specific operational realities.
+*   **Structured Learning:** Creates a clear, auditable trail of how and why procedures are evolving.
+*   **Analyst-Driven:** Empowers the security team to directly shape and improve the agent's capabilities without needing to edit the core runbooks.
+*   **Dynamic Expertise:** Transforms static runbooks into a dynamic knowledge base that represents our collective `Institutional Memory`.

rules_bank/institutional_memory/MEMORY-THESAURUS.md

@@ -0,0 +1,226 @@
+---
+generated: 2025-08-23T00:00:00Z
+directory: rules_bank/institutional_memory
+purpose: controlled_vocabulary
+extends: rules_bank/LLMS-THESAURUS.md
+---
+
+# Memory Framework Thesaurus for Institutional Learning
+
+## Notation Guide
+- **BT**: Broader Term (parent/category term)
+- **NT**: Narrower Term (child/specific term)  
+- **RT**: Related Term (associated but not hierarchical)
+- **SN**: Scope Note (clarification or usage guidance)
+- **USE FOR**: Preferred term (use this instead of variants)
+
+## Memory Framework Terms
+
+### ADAPTIVE LEARNING
+  USE FOR: Dynamic Learning, AI Learning, Procedural Evolution
+  BT: Artificial Intelligence
+  NT: Memory Creation, Memory Application, Memory Validation
+  RT: Institutional Memory, Feedback Processing, Confidence Scoring
+  SN: System capability to modify behavior based on operational experience
+
+### ANALYST FEEDBACK
+  USE FOR: Human Feedback, Operational Feedback, User Input
+  BT: Human-AI Interaction
+  NT: Procedural Feedback, Context Feedback, Performance Feedback
+  RT: Memory Creation, Feedback Processing, Validation
+  SN: Human analyst input used to improve AI procedural execution
+
+### APPLICATION LOG
+  USE FOR: Memory Log, Usage History, Execution Record
+  BT: Audit Trail
+  RT: Memory Validation, Performance Tracking, Confidence Adjustment
+  SN: Chronological record of memory application and outcomes
+
+### CONFIDENCE SCORING
+  USE FOR: Confidence Rating, Trust Score, Reliability Metric
+  BT: Performance Metrics
+  NT: High Confidence, Medium Confidence, Low Confidence
+  RT: Memory Validation, Success Rate, Application Criteria
+  SN: Numerical measure (0.0-1.0) of memory reliability and effectiveness
+
+### DERIVED PROCEDURE
+  USE FOR: Modified Procedure, Enhanced Procedure, Learned Procedure
+  BT: Procedural Knowledge
+  RT: Original Procedure, Memory Application, Institutional Learning
+  SN: Modified or new procedure created from institutional memory
+
+### FEEDBACK PROCESSING
+  USE FOR: Feedback Analysis, Input Processing, Learning Integration
+  BT: Data Processing
+  NT: Feedback Validation, Feedback Classification, Feedback Integration
+  RT: Analyst Feedback, Memory Creation, Meta-Agent Processing
+  SN: Systematic processing of human feedback into structured memories
+
+### INSTITUTIONAL MEMORY
+  USE FOR: Organizational Memory, Collective Knowledge, Learned Experience
+  BT: Knowledge Management
+  NT: Procedural Memory, Contextual Memory, Pattern Memory
+  RT: Adaptive Learning, Knowledge Base, Organizational Context
+  SN: Collective operational knowledge and learned procedures specific to an organization
+
+### MEMORY APPLICATION
+  USE FOR: Memory Usage, Procedure Enhancement, Dynamic Execution
+  BT: Procedural Execution
+  RT: Memory Query, Confidence Scoring, Validation
+  SN: Process of applying institutional memory to modify or enhance runbook procedures
+
+### MEMORY CREATION
+  USE FOR: Memory Formation, Knowledge Capture, Learning Integration
+  BT: Knowledge Acquisition
+  RT: Analyst Feedback, Feedback Processing, Memory Validation
+  SN: Process of transforming feedback and experience into structured memory files
+
+### MEMORY QUERY
+  USE FOR: Memory Lookup, Memory Search, Relevance Check
+  BT: Information Retrieval
+  RT: Memory Application, Procedural Context, Relevance Matching
+  SN: Process of searching for relevant memories before procedure execution
+
+### MEMORY RETIREMENT
+  USE FOR: Memory Deprecation, Knowledge Obsolescence, Memory Removal
+  BT: Knowledge Lifecycle
+  RT: Memory Validation, Expiration Date, Performance Metrics
+  SN: Process of removing outdated or ineffective memories from active use
+
+### MEMORY TYPE
+  USE FOR: Memory Classification, Memory Category, Learning Type
+  BT: Classification System
+  NT: Procedure Modification, Procedure Addition, Context Enhancement, Tool Substitution
+  RT: Memory Creation, Organizational Context, Application Criteria
+  SN: Standardized categories for classifying different types of institutional memories
+
+### MEMORY VALIDATION
+  USE FOR: Memory Testing, Effectiveness Assessment, Quality Control
+  BT: Quality Assurance
+  NT: Success Rate Tracking, Performance Measurement, Confidence Adjustment
+  RT: Application Log, Validation Metrics, Memory Retirement
+  SN: Process of verifying memory effectiveness and adjusting confidence scores
+
+### META-AGENT PROCESSING
+  USE FOR: Meta-Agent Analysis, Higher-Order Processing, Learning Orchestration
+  BT: Agent Architecture
+  RT: Feedback Processing, Memory Creation, System Learning
+  SN: Specialized AI agent responsible for processing feedback into structured memories
+
+### ORGANIZATIONAL CONTEXT
+  USE FOR: Organizational Specificity, Enterprise Context, Local Adaptation
+  BT: Contextual Knowledge
+  NT: Security Posture, Tool Environment, Process Requirements, Compliance Needs
+  RT: Institutional Memory, Procedural Adaptation, Environmental Factors
+  SN: Organization-specific factors that influence security operations and procedures
+
+### ORIGINAL PROCEDURE
+  USE FOR: Base Procedure, Standard Procedure, Default Workflow
+  BT: Procedural Knowledge
+  RT: Derived Procedure, Runbook Steps, Memory Enhancement
+  SN: Original runbook procedure before institutional memory modifications
+
+### PATTERN RECOGNITION
+  USE FOR: Pattern Detection, Recurring Patterns, Operational Patterns
+  BT: Data Analysis
+  NT: False Positive Patterns, Threat Patterns, Behavioral Patterns
+  RT: Pattern Memory, Institutional Learning, Context Analysis
+  SN: Identification of recurring operational patterns for procedural optimization
+
+### PERSONA-SPECIFIC ADAPTATION
+  USE FOR: Role-Based Customization, Persona Enhancement, Role-Specific Learning
+  BT: Personalization
+  RT: Security Personas, Memory Application, Contextual Adaptation
+  SN: Memory adaptations specific to security roles and responsibilities
+
+### PROCEDURAL EVOLUTION
+  USE FOR: Process Improvement, Workflow Evolution, Operational Enhancement
+  BT: Process Management
+  RT: Adaptive Learning, Institutional Memory, Continuous Improvement
+  SN: Ongoing improvement of procedures through institutional learning
+
+### SUCCESS RATE
+  USE FOR: Application Success Rate, Effectiveness Rate, Performance Rate
+  BT: Performance Metrics
+  RT: Confidence Scoring, Memory Validation, Application Log
+  SN: Ratio of successful memory applications to total attempts
+
+### VALIDATION COUNT
+  USE FOR: Application Count, Usage Count, Validation Attempts
+  BT: Performance Metrics
+  RT: Memory Validation, Confidence Scoring, Success Rate
+  SN: Number of times a memory has been successfully applied
+
+### VALIDATION METRICS
+  USE FOR: Success Indicators, Performance Indicators, Quality Measures
+  BT: Performance Metrics
+  NT: Success Rate, Response Time, Quality Improvement, Error Reduction
+  RT: Memory Validation, Confidence Scoring, Effectiveness Assessment
+  SN: Measurable indicators of memory effectiveness and procedural improvement
+
+## Memory Type Categories
+
+### COMPLIANCE_REQUIREMENT
+  BT: Memory Type
+  RT: Regulatory Compliance, Policy Adherence, Audit Requirements
+  SN: Memories capturing mandatory procedural changes for compliance
+
+### CONTEXT_ENHANCEMENT
+  BT: Memory Type
+  RT: Situational Awareness, Additional Context, Information Enrichment
+  SN: Memories that add contextual information to existing procedures
+
+### FALSE_POSITIVE_PATTERN
+  BT: Memory Type
+  RT: Pattern Recognition, Alert Filtering, Noise Reduction
+  SN: Memories capturing known benign patterns to reduce false positives
+
+### ORGANIZATIONAL_PREFERENCE
+  BT: Memory Type
+  RT: Organizational Context, Operational Preference, Process Customization
+  SN: Memories reflecting organization-specific operational preferences
+
+### PERFORMANCE_OPTIMIZATION
+  BT: Memory Type
+  RT: Efficiency Improvement, Speed Enhancement, Resource Optimization
+  SN: Memories focused on improving procedural efficiency and speed
+
+### PROCEDURE_ADDITION
+  BT: Memory Type
+  RT: Process Enhancement, Workflow Extension, Step Addition
+  SN: Memories that add new steps to existing procedures
+
+### PROCEDURE_MODIFICATION
+  BT: Memory Type
+  RT: Process Change, Workflow Alteration, Step Modification
+  SN: Memories that modify existing procedural steps
+
+### QUALITY_ENHANCEMENT
+  BT: Memory Type
+  RT: Analysis Quality, Investigation Depth, Result Accuracy
+  SN: Memories focused on improving analysis quality and thoroughness
+
+### TOOL_SUBSTITUTION
+  BT: Memory Type
+  RT: Tool Replacement, Alternative Tools, Technology Adaptation
+  SN: Memories specifying alternative tools for specific scenarios
+
+## Confidence Level Terms
+
+### HIGH_CONFIDENCE
+  USE FOR: High Trust, Proven Memory, Validated Learning
+  BT: Confidence Level
+  RT: Automatic Application, Validated Memory, High Success Rate
+  SN: Confidence level 0.9-1.0, suitable for automatic application
+
+### LOW_CONFIDENCE
+  USE FOR: Experimental Memory, Unproven Learning, Cautious Application
+  BT: Confidence Level
+  RT: Analyst Approval, Careful Monitoring, Validation Required
+  SN: Confidence level 0.0-0.3, requires analyst approval before application
+
+### MEDIUM_CONFIDENCE
+  USE FOR: Moderate Trust, Partially Validated, Recommended Application
+  BT: Confidence Level
+  RT: Recommended Procedure, Monitored Application, Confidence Building
+  SN: Confidence level 0.4-0.8, suitable for recommendation with explanation