Skip to content

Map MetaSkills eval templates to AgentV YAML specs #20

Description

@smota

Parent epic: #18

Goal

Document mapping from existing MetaSkills eval templates to AgentV YAML specs.

Existing source templates

  • skills/agent-builder/templates/agent-evals.md
  • skills/skill-creator/templates/skill-eval-spec.md

Mapping to define

MetaSkills fields:

  • Scenario
  • Prompt
  • Expected behavior
  • Must include
  • Must not include
  • Pass criteria
  • Category
  • Known gap / regression flags

AgentV concepts:

  • eval description
  • target/provider
  • tests
  • input
  • criteria
  • script graders
  • llm-rubric graders
  • result metadata/tags

Acceptance criteria

  • Add toolsets/agentv/mappings/metaskills-eval-to-agentv.md.
  • Include at least one full before/after conversion example.
  • Include guidance for deterministic checks vs LLM rubric checks.
  • Include category mapping for scope, knowledge, tool-action, safety, UX, orchestration, environment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestroadmapRoadmap planning and execution

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions