Skip to content

Add agentskills validate test to gws-cli recipe#45

Merged
pavelzw merged 1 commit intopavelzw:mainfrom
ytausch:yt/gws-agentskill-validate
Mar 25, 2026
Merged

Add agentskills validate test to gws-cli recipe#45
pavelzw merged 1 commit intopavelzw:mainfrom
ytausch:yt/gws-agentskill-validate

Conversation

@ytausch
Copy link
Copy Markdown
Contributor

@ytausch ytausch commented Mar 24, 2026

Summary

Test plan

  • CI passes with the new validation tests for all 93 gws-cli outputs

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests

    • Validation tests for installed skills are now automatically executed during the recipe build process, ensuring all skill installations conform to requirements. Enhanced testing pipeline with support for configurable runtime test dependencies.
  • Documentation

    • Removed outdated documentation sections regarding previously skipped validation checks, reflecting the updated testing methodology for skill packages.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 24, 2026

📝 Walkthrough

Walkthrough

The PR removes documentation about intentionally skipped validation from the README and implements per-skill validation testing in the gws-cli recipe generation pipeline, adding agentskills validate test entries and a skills-ref runtime dependency for each skill subpackage.

Changes

Cohort / File(s) Summary
Documentation
recipes/gws-cli/README.md
Removed "Skipped validation" section explaining why agentskills validate was intentionally skipped.
Test Generation Logic
recipes/gws-cli/generate.py
Added per-skill test generation that runs agentskills validate $CONDA_PREFIX/share/agent-skills/{skill} with a runtime dependency on skills-ref.
Generated Recipe Configuration
recipes/gws-cli/recipe.yaml
Auto-generated file now includes validation test entries for each skill subpackage alongside existing package contents checks.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • Add gws-cli recipe with generate script #33: Touches the same gws-cli generation pipeline; this PR builds upon that work by adding per-skill agentskills validate tests and skills-ref dependency to the generated outputs.

Suggested reviewers

  • pavelzw
  • moritzwilksch

Poem

🐰 Hop, skip, and validate!
Skills now tested at each gate,
No more skipping checks today,
Our fuzzy friend has found the way!


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error)

Check name Status Explanation Resolution
Recipe Version Bump ❌ Error Pull request modified recipe files (generate.py, recipe.yaml, README.md) but did not bump version or build number in recipe.yaml; version remains at upstream source version 0.22.0 with no recipe-specific build number. Add build_number field to recipe.yaml context section and update generate.py to include it in output, or update VERSION constant if upstream version changed.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding agentskills validate tests to the gws-cli recipe across multiple files.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@pavelzw
Copy link
Copy Markdown
Owner

pavelzw commented Mar 24, 2026

can you bump the build number to 1 to see how long recipe generation takes? creating 93 test environments might take quite a bit 😓

@ytausch
Copy link
Copy Markdown
Contributor Author

ytausch commented Mar 24, 2026

can you bump the build number to 1 to see how long recipe generation takes? creating 93 test environments might take quite a bit 😓

Locally, it was okay for me. I will do it once this is resolved:

Blocked by googleworkspace/cli#601

Re-introduce the skills-ref validation test for gws-cli packages.
This was previously skipped because upstream used JSON-style flow
sequences in SKILL.md frontmatter, which has since been fixed
(googleworkspace/cli#521, googleworkspace/cli#580).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ytausch ytausch force-pushed the yt/gws-agentskill-validate branch 2 times, most recently from 712920d to 86b4063 Compare March 25, 2026 12:17
@ytausch ytausch marked this pull request as ready for review March 25, 2026 12:17
@ytausch
Copy link
Copy Markdown
Contributor Author

ytausch commented Mar 25, 2026

@pavelzw Package build takes around 2 minutes: https://github.com/pavelzw/skill-forge/actions/runs/23540458859/job/68526394413?pr=45

(I temporarily bumped the build number)

@pavelzw pavelzw merged commit c1d6989 into pavelzw:main Mar 25, 2026
8 of 9 checks passed
@ytausch ytausch deleted the yt/gws-agentskill-validate branch March 25, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants