Issue #1085: Prefer AZD pattern modules first, fallback to AVM modules

[jongio]

Summary

This PR strengthens Azure guidance to prioritize module selection in the order that yields the best customer outcomes:

1. Use AZD pattern modules first when available
2. If no AZD pattern exists, use AVM resource modules
3. Then use AVM utility modules as needed

Why these changes

• AZD pattern modules are designed and validated for the Azure Developer CLI workflow, so they are more likely to deploy successfully end-to-end for customers.
• AVM modules provide a strong, supported fallback path when no AZD pattern module exists.
• This reduces trial-and-error in generated IaC and improves first-run success rates.

References

• Issue: Prefer AVM/AVM-AZD modules in skills and define integration path (Context7 vs Microsoft Learn MCP) #1085
• Azure Verified Modules (AVM): https://azure.github.io/Azure-Verified-Modules/
• AVM on Microsoft Learn: https://learn.microsoft.com/azure/azure-resource-manager/bicep/verified-modules
• Azure Developer CLI (azd): https://learn.microsoft.com/azure/developer/azure-developer-cli/

Scope

• Guidance updates in azure-prepare and azure-deploy references
• Added/updated deploy eval coverage for AVM/AZD ordering and fallback behavior
• Included related test harness compatibility fix already in branch history

[Copilot]

Pull request overview

Updates Azure skill guidance and evaluation coverage to prioritize AZD/AVM module selection in a way that aims to improve first-run deployment success.

Changes:

• Updated azure-prepare / azure-deploy skill descriptions and recipe references to emphasize AVM(+AZD) module selection order and fallback behavior.
• Added a committed Waza eval suite for azure-deploy (eval spec, trigger tests, and AVM ordering/fallback tasks).
• Adjusted the integration test agent runner to use a local approveAll permission handler compatible with the current Copilot SDK import surface.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/utils/agent-runner.ts	Replaces SDK approveAll import with a locally defined PermissionHandler implementation for sessions.
tests/azure-prepare/snapshots/triggers.test.ts.snap	Updates trigger keyword snapshots to reflect the new azure-prepare description text.
tests/azure-deploy/eval/trigger_tests.yaml	Adds deploy trigger routing prompts for the new committed azure-deploy eval suite.
tests/azure-deploy/eval/tasks/avm-order-bicep.yaml	Adds an eval task intended to validate AVM+AZD pattern-first guidance for Bicep deploy.
tests/azure-deploy/eval/tasks/avm-fallback-no-pattern.yaml	Adds an eval task intended to validate AVM fallback behavior when no pattern module exists.
tests/azure-deploy/eval/eval.yaml	Adds the main Waza eval spec for azure-deploy with metrics/graders and task globbing.
tests/azure-deploy/snapshots/triggers.test.ts.snap	Updates trigger keyword snapshots to reflect the new azure-deploy description text.
tests/README.md	Documents azure-deploy as a committed eval suite in the hybrid Waza model.
plugin/skills/azure-prepare/references/recipes/bicep/README.md	Links standalone Bicep recipe module guidance to the AVM module ordering rules.
plugin/skills/azure-prepare/references/recipes/azd/terraform.md	Adds Terraform-specific AVM module priority guidance and retrieval strategy notes.
plugin/skills/azure-prepare/references/recipes/azd/iac-rules.md	Introduces AVM module ordering rules plus a stated docs retrieval/validation plan.
plugin/skills/azure-prepare/references/recipes/azd/README.md	Links AZD recipe AVM guidance to the module ordering section.
plugin/skills/azure-prepare/references/recipes/azcli/README.md	Links azcli recipe module guidance to the AVM module ordering rules.
plugin/skills/azure-prepare/SKILL.md	Refines the skill description to focus on “prepare” scenarios and updates trigger phrasing.
plugin/skills/azure-deploy/references/recipes/bicep/README.md	Adds an AVM verification checklist prior to running deployment commands.
plugin/skills/azure-deploy/SKILL.md	Refines the skill description and fixes formatting around the references list.

[Copilot]

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

[jongio]

Addressed the two latest Copilot review threads in commit a4d4d95:

• Updated azure-prepare frontmatter wording to avoid migration-trigger overlap and use explicit on-premises phrasing while keeping cross-cloud routing to azure-cloud-migrate.
• Updated azd iac-rules fallback wording to be provider-scoped: if no pattern module exists for the active provider, fallback remains AVM resource -> AVM utility for that provider.

(Inline thread replies via GraphQL addPullRequestReviewComment were blocked by permission in this fork context, so I’m posting confirmation here.)

[Copilot]

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

plugin/skills/azure-prepare/SKILL.md:58

• The Step 0 routing table omits the highest-priority cross-cloud/AWS/GCP migration routing to azure-cloud-migrate (even though the skill description says cross-cloud should use azure-cloud-migrate). Since Step 0 is the “MANDATORY FIRST ACTION”, leaving this out can cause incorrect first-skill invocation. Add the azure-cloud-migrate row here (or explicitly point to the full routing table as the source of truth, including that row).

**BEFORE starting Phase 1**, check if the user's prompt mentions a specialized technology that has a dedicated skill with tested templates. If matched, **invoke that skill FIRST** — then resume azure-prepare for validation and deployment.

| Prompt keywords | Invoke FIRST |
|----------------|-------------|
| copilot SDK, copilot app, copilot-powered, @github/copilot-sdk, CopilotClient | **azure-hosted-copilot-sdk** |
| Azure Functions, function app, serverless function, timer trigger, HTTP trigger, func new | Stay in **azure-prepare** — prefer Azure Functions templates in Step 4 |
| APIM, API Management, API gateway, deploy APIM | Stay in **azure-prepare** — see [APIM Deployment Guide](references/apim.md) |
| AI gateway, AI gateway policy, AI gateway backend, AI gateway configuration | **azure-aigateway** |

[jongio]

We got a little carried away with trying to fix everything in this PR, so I'm just going to trim it back down to AVM changes only.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.