Skip to content

applies-to-tagging: add generate-from-intent mode and cumulative-docs rules#69

Merged
theletterf merged 5 commits into
elastic:mainfrom
florent-leborgne:applies-to-tagging-generate-and-preservation
May 21, 2026
Merged

applies-to-tagging: add generate-from-intent mode and cumulative-docs rules#69
theletterf merged 5 commits into
elastic:mainfrom
florent-leborgne:applies-to-tagging-generate-and-preservation

Conversation

@florent-leborgne

Copy link
Copy Markdown
Member

Summary

Two related additions that make docs-applies-to-tagging the canonical home for both applies_to syntax and the cumulative-docs decisions that surround it (preserve vs. replace, append vs. replace lifecycle, when to delete a removed feature's page).

  • Generate-from-intent mode — accept a structured description of a change and return the right applies_to syntax, instead of only validating an existing file.
  • Cumulative documentation rules — capture the preservation logic that determines whether new tagged content should sit alongside old content or replace it.

4 new evals exercise both additions. Skill version: 1.1.0 → 1.2.0 (MINOR). Plugin version: 1.0.3 → 1.0.4.

Why

Two gaps exist today:

  1. The skill says it can "generate correct ones for new or updated content," but the only documented entry point for generation is "if asked to fix or generate tags, use Edit to apply corrections" — i.e., file-bound. Callers that have a structured change description but no file (for example, the docs-quest-scanner skill, when drafting a docsGap entry from a PR) have no clean way to ask this skill to generate a tag.
  2. The cumulative-docs preservation rules (do users on older versions still need the old content? when do you append a lifecycle vs. replace it? when do you keep a removed feature's page?) live informally inside docs-quest-scanner's SKILL.md instead of next to the syntax they govern.

This PR fixes both, so downstream skills can delegate to one canonical source.

What changed

SKILL.md

Modes section (new)

Distinguishes validate mode (existing — file or pasted content) from generate-from-intent mode (new — structured change description). Detection cues for the new mode are explicit; ambiguous cases prompt one focused clarifying question rather than guessing.

Cumulative documentation rules (new section, before Task execution)

  • Preserve old content when possible — two questions to ask before any version-scoped change.
  • Lifecycle changes — versioned products append (stack: ga 9.2+, preview =9.0), unversioned replace.
  • Removals — GA/deprecated removed from versioned product → keep content + removed 9.x; removed from unversioned only → can delete; preview/beta-only feature ending → can delete.
  • Lightest-format preference — tagged paragraph or admonition → tagged list items → applies-switch tabs (last resort).
  • Version display reminders — no version numbers in prose adjacent to badges; Major.Minor display; latest-patch semantics.

Generate-from-intent execution (new section)

Five-step flow:

  1. Extract the change description (dimension, lifecycle per key, version per lifecycle, sub-projects, scope, preserve-vs-replace).
  2. Decide preservation vs. replacement using the cumulative rules.
  3. Pick the lightest format that scopes correctly.
  4. Generate the canonical syntax (frontmatter, fenced section block, inline, admonition).
  5. Output with explicit rationale and a preservation note when applicable.

Task execution (lightly edited)

The validate flow now references generate-from-intent for the "no file, just a change description" case so users land in the right mode.

Description (frontmatter)

Widened to mention preservation decisions — this is now a triggering use case in addition to validation and generation.

evals.json

4 new evals (ids 6–9):

  • 6 — Generate-from-intent, simple stack + serverless: feature added in 9.5 in stack and in serverless elasticsearch. Expects stack: ga 9.5+ plus the serverless.elasticsearch: ga sub-key (not bare serverless: ga, since the user specified the sub-project).
  • 7 — Generate-from-intent, multi-lifecycle progression: stack-only feature, preview at 9.4, GA at 9.5. Expects stack: ga 9.5+, preview =9.4, ordered newest-first, with =9.4 for the exact preview version.
  • 8 — Cumulative preservation, default-value change: a configuration default changed in 9.4. Expects "preserve the existing paragraph, add a tagged paragraph or admonition for the new default" — not replacement.
  • 9 — Cumulative preservation, removal from versioned product: GA in stack until 9.3, removed in 9.4. Expects "keep the page, append stack: removed 9.4," not deletion.

Test plan

  • python3 -m json.tool evals/evals.json — valid
  • Local replay of validate-skills.yml Python checks — 17 skills validated, 0 errors
  • Manual review: existing validation rules and syntax sections unchanged; existing evals 1–5 unchanged
  • CI validate-skills.yml and skill-eval-test.md runs on PR
  • Eval grading on the 4 new cases

Notes for reviewers

  • Ordering on the page: I put the new Cumulative documentation rules before Generate-from-intent execution because the generation flow references those rules (Step 2 explicitly applies them). They could equally live as a subsection of the generation flow, but cumulative rules also apply during validation and as standalone questions, so I gave them their own section.
  • Plugin version bump conflicts with another open PR (content-type-checker: add classify-proposal mode #68content-type-checker). Whichever lands second will need a one-line plugin version rebase.
  • The existing language "validate existing applies_to tags and generate correct ones for new or updated content" already promised generation; this PR simply makes the generation path first-class with a real entry point and rationale.

florent-leborgne and others added 5 commits May 8, 2026 11:22
… rules

Two related additions to make this skill the canonical home for both
applies_to syntax *and* the cumulative-docs decisions that surround it
(preserve vs replace, append vs replace lifecycle, when to delete a
removed feature's page).

1) Generate-from-intent mode
   The skill currently only validates existing tags or fixes them in
   place. Callers who need to generate the right tag from a structured
   change description (feature, version, lifecycle, dimension,
   products) — for example, the docs-quest-scanner skill when drafting
   a docsGap entry from a PR — have no clean entry point. Adds a
   second mode with a five-step flow: extract change description →
   apply preservation rules → pick the lightest format → generate
   syntax → output with rationale.

2) Cumulative documentation rules
   Captures the "do users on previous versions still need the old
   information?" question and its consequences:
   - Versioned products (stack): append the new lifecycle, keep the old
   - Unversioned products (serverless): replace the old state
   - Removals from versioned products: keep content + add `removed`
   - Lightest-format preference order: tagged paragraph/admonition →
     tagged list items → applies-switch tabs (last resort)

   These rules already live (informally) inside the docs-quest-scanner
   skill. Lifting them here puts them next to the syntax they govern.

Changes:
- SKILL.md: adds Modes section, Cumulative-documentation-rules section
  (between Common patterns and Task execution), Generate-from-intent
  execution section. Existing validation rules and syntax reference
  unchanged.
- evals.json: 4 new evals (ids 6–9) covering simple generation
  (stack+serverless), multi-lifecycle generation (preview→GA), default
  value preservation, and removal handling.
- Skill version bumped 1.1.0 → 1.2.0 (MINOR — new capabilities, no
  breaking change).
- Plugin version bumped 1.0.3 → 1.0.4 per validate-skills CI.

Motivation: docs-quest-scanner currently inlines both a long
applies_to syntax cheatsheet and the cumulative-docs decision rules.
Lifting these upstream lets the orchestrator skill delegate cleanly
and keeps one canonical version of the rules.
Replaces the "Lightest-format preference" framing with the analytical
principle from docs-content/CLAUDE.md: it's not about lightness, it's
about scope. The new framing has three pieces:

1. Decide whether applies_to is needed at all
   - Don't tag when the edit is valid for all versions (rewording,
     typo fixes, restructuring)
   - Don't tag when a parent page or section already has the correct
     applies_to
   - Don't tag when a small inline pattern carries the meaning (most
     commonly a renamed UI element written as
     "Select **New name** (or **Old name** in earlier versions)")
   - Don't tag GA features in unversioned products where the
     page-level lifecycle already covers it
   - Tag when content is genuinely version- or deployment-scoped,
     isn't covered by a parent tag, and isn't better expressed inline

2. Place applies_to where the change applies
   - Section level (fenced block after heading) when the change is
     relevant to a section
   - Page level (frontmatter) when the whole page is scoped
   - Inline only at start of list item / end of definition term /
     inside table cell — never mid-sentence or floating between
     sentences in a paragraph
   - Restructure prose into an admonition or dropdown with
     `:applies_to:` when it doesn't fit those inline positions
   - applies-switch tabs only when content truly diverges

3. Scope must be unambiguous
   - Reader must be able to tell exactly what the tag covers from
     heading + block position alone

Also drops the misframed "Lightest-format preference" subsection
from Cumulative documentation rules and updates the
generate-from-intent Step 3 wording to reference the corrected
placement rules.

Adds eval id 10: a renamed-UI-element prompt that should NOT result
in an applies_to tag, but in the inline parenthetical pattern. This
exercises the gating-rule carve-out that's easy to get wrong.
Trivial conflict: — escape vs literal — em dash in description.
Kept the literal character form from main.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@theletterf theletterf merged commit 9463105 into elastic:main May 21, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants