Skip to content

Surface deprecated prebuilt detection rules as a sibling nav page#3167

Open
Mpdreamz wants to merge 9 commits intomainfrom
feature/deprecated-detection-rules
Open

Surface deprecated prebuilt detection rules as a sibling nav page#3167
Mpdreamz wants to merge 9 commits intomainfrom
feature/deprecated-detection-rules

Conversation

@Mpdreamz
Copy link
Copy Markdown
Member

@Mpdreamz Mpdreamz commented Apr 22, 2026

Why

Elastic's prebuilt detection rules repo contains a _deprecated/ folder of rules that have been retired. These rules were previously invisible to users browsing the docs — there was no way to discover them, understand why they exist, or know what replaced them. Users navigating directly to a deprecated rule page (e.g. from a bookmark or external link) also lost their nav context entirely: nothing in the sidebar was highlighted.

What

Deprecated rules page

When the detection-rules plugin processes a rule folder that contains a _deprecated/ subfolder, it automatically generates a sibling "Deprecated prebuilt detection rules" page in the navigation alongside the active rules overview. No docset.yml changes are required for the page to appear.

The page lists all deprecated rules alphabetically by domain, linking to individual rule pages. Each individual rule page shows a deprecation warning admonition with the date the rule was deprecated.

Optional preface

The security team can supply introductory content for the deprecated rules page by adding a markdown file to the detection-rules docs folder and referencing it in docset.yml:

- file: index.md
  detection_rules: ['../rules', '../rules_building_block']
  deprecated_file: deprecated-rules.md   # optional

When deprecated_file is set, the content of that file prefixes the auto-generated rules listing. This lets the team explain what deprecated rules are, what users should do with them, and link to related guidance — all without touching the docs-builder codebase.

Follow-up: elastic/detection-rules#5968 adds this preface file (draft, cannot merge until this PR is released to production).

Nav active state for hidden pages

Individual rule pages (both active and deprecated) are hidden nav leaves — they exist in the nav tree for ordering/breadcrumb purposes but don't appear as visible links in the sidebar. Before this change, landing on a rule page left the sidebar with nothing selected.

The fix emits a <meta name="docs:nav-active" content="{parentUrl}"> tag server-side when the current page is a hidden nav item. The client reads this and highlights the nearest visible ancestor instead. This restores nav context for rule pages and fixes the same issue on the live site for any hidden page.

Verification

Run locally against the detection-rules repo:

dotnet run --project src/tooling/docs-builder -- serve --path /path/to/detection-rules
  • http://localhost:3000/deprecated-detection-rules — lists deprecated rules
  • http://localhost:3000/rules/_deprecated/<rule> — deprecation warning, sidebar highlights deprecated rules page
  • http://localhost:3000/rules/<category>/<rule> — sidebar highlights prebuilt rules overview

Mpdreamz and others added 2 commits April 22, 2026 20:52
- Auto-detects _deprecated/ subfolders in detection rule folders and generates
  a sibling "Deprecated prebuilt detection rules" page in the nav
- Optional deprecated_file: field in docset.yml names a .md file whose content
  prefixes the auto-generated deprecated rules listing
- Each deprecated rule page shows a deprecation warning with the deprecation date
- Fixed stateful TomlParser causing serve-mode parse failures on repeated builds
- Fixed serve --path scope: in-memory write FS now scoped to git root of --path
  so output at {externalRepo}/.artifacts is within allowed write scope
- Fixed serve sourcePath: ReloadGeneratorService now passes DocumentationCheckoutDirectory
  (git root) rather than DocumentationSourceDirectory (docs/ subfolder)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
When a page is a hidden nav leaf (e.g. individual detection rule pages),
the JavaScript active-state logic couldn't find a matching nav link and
left the sidebar with nothing selected.

Fix: emit <meta name="docs:nav-active" content="{parentUrl}"> in the
page <head> when current.Hidden is true, pointing to the nearest visible
ancestor. pages-nav.ts reads this tag and uses that URL to mark the
correct nav item as current instead of window.location.pathname.

This works for both deprecated rules (/deprecated-detection-rules parent)
and active prebuilt rules (/ parent = detection rules overview at index.md).
Visible pages are unaffected (no meta tag emitted).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@Mpdreamz Mpdreamz requested a review from a team as a code owner April 22, 2026 19:43
@Mpdreamz Mpdreamz requested a review from technige April 22, 2026 19:43
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 22, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d1fe92bb-39b0-42a3-bc27-1ac524890d16

📥 Commits

Reviewing files that changed from the base of the PR and between 9141470 and ba70441.

📒 Files selected for processing (3)
  • Directory.Packages.props
  • src/Elastic.Documentation.Configuration/FileSystemFactory.cs
  • src/Elastic.Markdown/HtmlWriter.cs
🚧 Files skipped from review as they are similar to previous changes (2)
  • Directory.Packages.props
  • src/Elastic.Documentation.Configuration/FileSystemFactory.cs

📝 Walkthrough

Walkthrough

Adds deprecated-detection-rules support across the docs pipeline: a scoped in-memory filesystem anchored to the repository root; TOC parsing and YAML converters that detect _deprecated rule folders and read an optional deprecated_file override; generation of deprecated overview markdown (physical or synthetic) and a sibling TOC ref; detection-rule TOML parsing migrated to Tomlyn and extended to include Maturity and DeprecationDate; detection-rule rendering emits a deprecation warning for deprecated rules; renderer and layout now emit a docs:nav-active meta tag (via NavigationActiveUrl) so frontend JS can mark the appropriate nav item for hidden pages; tests for TOML parsing added.

Sequence Diagram(s)

sequenceDiagram
    participant Builder as Build System
    participant TOC as TOC Resolver
    participant FS as InMemory FileSystem
    participant Disk as Disk
    participant Parser as Tomlyn Parser
    participant Renderer as HTML Renderer
    participant Browser as Browser/JS

    Builder->>FS: InMemoryForSourceRoot(sourcePath)
    Builder->>TOC: ResolveTableOfContents()
    TOC->>Disk: Scan DetectionRuleFolders
    Disk-->>TOC: entries (include _deprecated?)
    alt _deprecated present
        TOC->>TOC: Create DeprecatedSiblingRef (deprecated overview FileRef)
        TOC->>Builder: Register deprecated overview (physical or synthetic)
    end
    Builder->>Parser: Parse .toml detection rule files
    Parser-->>Builder: DetectionRule (includes Maturity, DeprecationDate)
    Builder->>Renderer: Render pages (insert deprecation warning if Maturity=deprecated)
    Renderer->>Renderer: Compute NavigationActiveUrl for hidden items
    Renderer->>Browser: Emit docs:nav-active meta tag
    Browser->>Browser: JS reads meta tag and marks nav item current
Loading

Suggested labels

feature, docs, tests

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 13.46% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely summarizes the primary change: surfacing deprecated prebuilt detection rules as a sibling navigation page, matching the core objective of making deprecated rules discoverable and maintaining nav context.
Description check ✅ Passed The description is directly related to the changeset, providing clear context on why the change was made, what was implemented (deprecated rules page, optional preface, nav active state for hidden pages), and verification steps.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch feature/deprecated-detection-rules

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 0/1 reviews remaining, refill in 60 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs (1)

75-87: ⚠️ Potential issue | 🟡 Minor

Remove unused deprecated_detection_rules from parsing logic.

The deprecated_detection_rules key is parsed into the dictionary but never read anywhere in the codebase. The PR auto-discovers deprecated rules via folder scanning, making this key dead code. Remove it from the condition or document why it's retained.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs`
around lines 75 - 87, In the YAML parsing branch inside
TableOfContentsYamlConverters (the else-if that checks key.Value against
"detection_rules" or "exclude" or "deprecated_detection_rules"), remove
"deprecated_detection_rules" from that condition and any related parsing branch
so the parser no longer adds that key into the dictionary; if you intentionally
want to keep it, add a comment explaining why and where it's consumed. Update or
remove any tests or usages that expect deprecated_detection_rules to be present
and ensure the parsing of the remaining keys ("detection_rules", "exclude") is
unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In
`@src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs`:
- Around line 75-87: In the YAML parsing branch inside
TableOfContentsYamlConverters (the else-if that checks key.Value against
"detection_rules" or "exclude" or "deprecated_detection_rules"), remove
"deprecated_detection_rules" from that condition and any related parsing branch
so the parser no longer adds that key into the dictionary; if you intentionally
want to keep it, add a comment explaining why and where it's consumed. Update or
remove any tests or usages that expect deprecated_detection_rules to be present
and ensure the parsing of the remaining keys ("detection_rules", "exclude") is
unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 81e46472-e23f-4459-8cba-6b14b17737ff

📥 Commits

Reviewing files that changed from the base of the PR and between b94fd12 and 9042a9a.

📒 Files selected for processing (15)
  • src/Elastic.Documentation.Configuration/FileSystemFactory.cs
  • src/Elastic.Documentation.Configuration/Toc/DetectionRules/DetectionRuleOverviewRef.cs
  • src/Elastic.Documentation.Configuration/Toc/DocumentationSetFile.cs
  • src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs
  • src/Elastic.Documentation.Site/Assets/pages-nav.ts
  • src/Elastic.Documentation.Site/Layout/_Head.cshtml
  • src/Elastic.Documentation.Site/_ViewModels.cs
  • src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs
  • src/Elastic.Markdown/Extensions/DetectionRules/DetectionRuleFile.cs
  • src/Elastic.Markdown/Extensions/DetectionRules/DetectionRulesDocsBuilderExtension.cs
  • src/Elastic.Markdown/HtmlWriter.cs
  • src/Elastic.Markdown/Page/Index.cshtml
  • src/Elastic.Markdown/Page/IndexViewModel.cs
  • src/tooling/docs-builder/Http/InMemoryBuildState.cs
  • src/tooling/docs-builder/Http/ReloadGeneratorService.cs

GHSA-g94r-2vxg-569j  OpenTelemetry.Api 1.13.1
GHSA-mr8r-92fq-pj8p  OpenTelemetry.Exporter.OpenTelemetryProtocol 1.13.1
GHSA-q834-8qmm-v933  OpenTelemetry.Exporter.OpenTelemetryProtocol 1.13.1

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
cotti
cotti previously approved these changes Apr 30, 2026
@cotti cotti self-requested a review April 30, 2026 03:52
@cotti cotti dismissed their stale review April 30, 2026 03:53

Saw an issue with TOML parsing unrelated to these changes, but still affecting parsing as a whole

@cotti
Copy link
Copy Markdown
Contributor

cotti commented Apr 30, 2026

I sent #3214 as a possible fix - it works better as a PR for this branch.

coderabbitai[bot]
coderabbitai Bot previously requested changes Apr 30, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs (1)

292-293: 💤 Low value

GetString throws unhelpful exception on missing keys.

Direct indexing table[key] throws KeyNotFoundException without context about which field is missing. For better debuggability in malformed TOML, consider wrapping with a clearer message.

Optional improvement
-private static string GetString(TomlTable table, string key) =>
-    (string)table[key];
+private static string GetString(TomlTable table, string key) =>
+    table.TryGetValue(key, out var value) && value is string s
+        ? s
+        : throw new InvalidOperationException($"Required field '{key}' is missing or not a string");
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs` around lines
292 - 293, The GetString helper currently indexes TomlTable directly causing a
KeyNotFoundException with no context; update GetString (the private static
method) to first check for the key (e.g. ContainsKey or TryGetValue) and if
missing throw an ArgumentException or KeyNotFoundException with a clear message
that includes the missing key name (and optionally the table name/context) and,
when present, safely convert the value to string (with null checks) so malformed
or null values produce a descriptive error instead of an unhelpful exception.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs`:
- Around line 281-290: ReadTactic currently indexes threatTable["tactic"] and
will throw if the key is missing; change it to use
threatTable.TryGetValue("tactic", out var t) and ensure t is a TomlTable before
casting, returning null (or a safe default) if the key is absent or not a
TomlTable; then call GetString(tacticTable, "...") only when tacticTable is
non-null so DetectionRuleTactic construction and GetString calls are guarded
against missing or malformed tactic entries.

---

Nitpick comments:
In `@src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs`:
- Around line 292-293: The GetString helper currently indexes TomlTable directly
causing a KeyNotFoundException with no context; update GetString (the private
static method) to first check for the key (e.g. ContainsKey or TryGetValue) and
if missing throw an ArgumentException or KeyNotFoundException with a clear
message that includes the missing key name (and optionally the table
name/context) and, when present, safely convert the value to string (with null
checks) so malformed or null values produce a descriptive error instead of an
unhelpful exception.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 87f9e368-6350-4e26-af0d-9198120ce3a6

📥 Commits

Reviewing files that changed from the base of the PR and between 9042a9a and 256f308.

📒 Files selected for processing (7)
  • Directory.Packages.props
  • src/Elastic.Documentation.Configuration/Elastic.Documentation.Configuration.csproj
  • src/Elastic.Documentation.Configuration/Toc/DocumentationSetFile.cs
  • src/Elastic.Markdown/Elastic.Markdown.csproj
  • src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs
  • src/Elastic.Markdown/HtmlWriter.cs
  • tests/Elastic.Markdown.Tests/DetectionRules/DetectionRuleParsingTests.cs
✅ Files skipped from review due to trivial changes (1)
  • src/Elastic.Documentation.Configuration/Elastic.Documentation.Configuration.csproj
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/Elastic.Markdown/HtmlWriter.cs

Comment thread src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants