Skip to content

ROADMAP.md

Latest commit

 

History

History
95 lines (64 loc) · 2.68 KB

ROADMAP.md

File metadata and controls

95 lines (64 loc) · 2.68 KB

Roadmap

This document outlines planned features for upcoming RAGLeakLab releases.

Note

This roadmap is subject to change based on community feedback and priorities. For proposing new features, see RFC.md.

v1.0.0 — Released ✅

The first stable release with a complete security testing toolkit.

Shipped Features

  • Five leakage threat packs (canary, verbatim, membership, semantic, cross-document)
  • Corpus poisoning detection (sentinel-takeover-safe pack)
  • CI regression gates (diff command)
  • Delta ingestion gates (corpus change detection)
  • SARIF + JUnit + Markdown output formats
  • Determinism verification (verify determinism)
  • Cassette record/replay for HTTP targets
  • Benchmark bundles (bench bundle / bench publish)
  • Threshold calibration (calibrate command)
  • Secret redaction (emails, API keys, canary tokens)
  • Parallel execution (--jobs N)
  • Query minimization (--minimize-on-fail)
  • Plugin system (entry-point based)
  • SSRF protection and domain allowlisting for HTTP targets
  • Asset validation (assets validate)
  • Config validation with JSON Schema export
  • Docker support

v1.1.0 — Semantic Leakage Expansion

Target: Q2 2026

Focus on deepening semantic leakage detection and improving claim taxonomy.

Features

  • Extended semantic claim taxonomy (financial, medical, legal, PII)
  • Claim confidence scoring improvements
  • Semantic pack v2 with 80+ test cases
  • Improved attribution for semantic leaks

Improvements

  • Faster claim matching with caching
  • Better false-positive filtering
  • Enhanced SARIF output for semantic findings

v1.2.0 — Advanced Membership Inference

Target: Q3 2026

Advanced membership inference with statistical rigor.

Features

  • Shadow model-based membership inference
  • Calibrated confidence scores with p-values
  • Differential privacy measurement
  • Per-document sensitivity scoring

Improvements

  • Reduced false positive rate (<1%)
  • Support for larger corpora (10k+ documents)
  • Parallel membership testing

v2.0.0 — Multi-Modal & Streaming

Target: 2027

Features

  • Multi-modal support: image/audio in RAG pipelines
  • Streaming detection: real-time leakage monitoring
  • Policy engine: define allowed/forbidden disclosures
  • LLM provider adapters: OpenAI, Anthropic, local models
  • Differential testing: compare RAG configurations

Contributing

Have ideas for the roadmap? Open a discussion, file an RFC, or check CONTRIBUTING.md.