-
Notifications
You must be signed in to change notification settings - Fork 92
Add comprehensive test suites, CI workflows, and Copilot instructions #602
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
918e45c
fa129c2
719c557
fdbf493
6937524
007a536
fd55096
a80956e
6e70e50
2b04fb4
2a92d8b
f9fb18e
5a2bcc0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,205 @@ | ||||||
| # Copilot Instructions for GitHub Copilot for Azure | ||||||
|
|
||||||
| ## Repository Overview | ||||||
|
|
||||||
| This repository contains agent skills for GitHub Copilot's Azure integration. Skills provide specialized capabilities for deployment, validation, diagnostics, and Azure service management. | ||||||
|
|
||||||
| --- | ||||||
|
|
||||||
| ## Directory Structure | ||||||
|
|
||||||
| ``` | ||||||
| plugin/ | ||||||
| └── skills/ | ||||||
| └── <skill-name>/ | ||||||
| ├── SKILL.md # Skill definition (required) | ||||||
| └── reference/ # Reference guides (optional) | ||||||
|
|
||||||
| tests/ | ||||||
| ├── detection/ # App type detection tests (264 tests) | ||||||
| │ ├── __tests__/ | ||||||
| │ ├── fixtures/ | ||||||
| │ ├── src/ | ||||||
| │ └── package.json | ||||||
| ├── nodejs-production/ # Node.js production readiness tests (83 tests) | ||||||
| │ ├── __tests__/ | ||||||
| │ ├── fixtures/ | ||||||
| │ ├── src/ | ||||||
| │ └── package.json | ||||||
| ├── validation/ # Azure validation tests (165 tests) | ||||||
| │ ├── __tests__/ | ||||||
| │ ├── fixtures/ | ||||||
| │ ├── src/ | ||||||
| │ └── package.json | ||||||
| └── README.md | ||||||
|
|
||||||
| .github/ | ||||||
| ├── agents/ # Custom agent definitions | ||||||
| ├── workflows/ # CI/CD workflows | ||||||
| └── ISSUE_TEMPLATE/ | ||||||
| ``` | ||||||
|
|
||||||
| --- | ||||||
|
|
||||||
| ## Skill Development | ||||||
|
|
||||||
| ### Skill File Format | ||||||
|
|
||||||
| Every skill requires a `SKILL.md` file with YAML frontmatter: | ||||||
|
|
||||||
| ```markdown | ||||||
| --- | ||||||
| name: skill-name | ||||||
| description: Brief description of what the skill does. Include trigger phrases. | ||||||
| --- | ||||||
|
|
||||||
| ## Skill content and instructions... | ||||||
| ``` | ||||||
|
|
||||||
| ### Key Guidelines | ||||||
|
|
||||||
| - **Use `azd` for deployments** - Azure Developer CLI, not `az` CLI | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We might need stronger guidance to use approved templates documented in skills and MCP tools first. but ok to wait on that. |
||||||
| - **Reference guides** - Place detailed service-specific docs in `reference/` subdirectory | ||||||
| - **MCP tools** - Document which Azure MCP tools the skill uses | ||||||
| - **Detection logic** - If the skill involves app detection, follow patterns in `azure-deploy` | ||||||
|
|
||||||
| --- | ||||||
|
|
||||||
| ## Testing | ||||||
|
|
||||||
| ### Test Structure | ||||||
|
|
||||||
| Tests are organized into focused suites, each with its own `package.json`: | ||||||
|
|
||||||
| | Suite | Location | Tests | Purpose | | ||||||
| |-------|----------|-------|---------| | ||||||
| | **Detection** | `tests/detection/` | 264 | App type detection, service selection | | ||||||
| | **Node.js Production** | `tests/nodejs-production/` | 83 | Production readiness validation | | ||||||
| | **Validation** | `tests/validation/` | 165 | Azure resource validation, Bicep, preflight | | ||||||
|
|
||||||
| ### Running Tests | ||||||
|
|
||||||
| ```bash | ||||||
| # Detection tests | ||||||
| cd tests/detection && npm test | ||||||
|
|
||||||
| # Node.js production tests | ||||||
| cd tests/nodejs-production && npm test | ||||||
|
|
||||||
| # Validation tests | ||||||
| cd tests/validation && npm test | ||||||
| ``` | ||||||
|
|
||||||
| ### Test Expectations | ||||||
|
|
||||||
| - **Detection tests**: 264+ tests, all should pass | ||||||
| - **Node.js production tests**: 83+ tests | ||||||
| - **Validation tests**: 165+ tests | ||||||
|
|
||||||
| --- | ||||||
|
|
||||||
| ## Adding Tests | ||||||
|
|
||||||
| ### ⚠️ IMPORTANT: Only Add Tests to Existing Suites | ||||||
|
|
||||||
| **DO NOT create new test suites or test areas.** Add tests only to existing coverage areas: | ||||||
|
|
||||||
| - `tests/__tests__/detection/` - Detection logic tests | ||||||
|
||||||
| - `tests/__tests__/detection/` - Detection logic tests | |
| - `tests/detection/__tests__/` - Detection logic tests |
Copilot
AI
Jan 28, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lines 168-172 show incorrect commands for running test suites. The commands reference just cd tests && npm test but the tests are organized into separate suites (detection, nodejs-production, validation) each with their own package.json. The correct commands should be cd tests/detection && npm test, etc., as shown correctly in lines 82-90.
| cd tests && npm test | |
| cd tests/detection && npm test |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| name: Detection Tests | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - 'plugin/skills/azure-deploy/**' | ||
| - 'tests/detection/**' | ||
| pull_request: | ||
| branches: [main] | ||
| paths: | ||
| - 'plugin/skills/azure-deploy/**' | ||
| - 'tests/detection/**' | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| test: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| strategy: | ||
| matrix: | ||
| node-version: [20, 22] | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. at some point later might need to move this to an overall config for actions/workflows so the runners consistently use the rright latest versions. |
||
|
|
||
| defaults: | ||
| run: | ||
| working-directory: tests/detection | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| cache: 'npm' | ||
| cache-dependency-path: tests/detection/package-lock.json | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci || npm install | ||
|
|
||
| - name: Run tests | ||
| run: npm test | ||
|
|
||
| - name: Run tests with coverage | ||
| run: npm run test:coverage | ||
|
|
||
| - name: Upload coverage to Codecov | ||
| if: matrix.node-version == 20 | ||
| uses: codecov/codecov-action@v4 | ||
| with: | ||
| directory: tests/detection/coverage | ||
| flags: detection | ||
| fail_ci_if_error: false | ||
| env: | ||
| CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
|
|
||
| lint: | ||
Check warningCode scanning / CodeQL Workflow does not contain permissions Medium test
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
|
||
| name: Lint | ||
| runs-on: ubuntu-latest | ||
|
|
||
| defaults: | ||
| run: | ||
| working-directory: tests/detection | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20 | ||
| cache: 'npm' | ||
| cache-dependency-path: tests/detection/package-lock.json | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci || npm install | ||
|
|
||
| - name: Check fixture/expectation sync | ||
| run: | | ||
| node -e " | ||
| const { validateFixturesHaveExpectations } = require('./utils/fixture-loader'); | ||
| const result = validateFixturesHaveExpectations(); | ||
| if (!result.valid) { | ||
| if (result.missing.length) console.error('Missing expectations:', result.missing); | ||
| if (result.extra.length) console.error('Extra expectations:', result.extra); | ||
| process.exit(1); | ||
| } | ||
| console.log('✓ All fixtures have expectations'); | ||
| " | ||
Check warningCode scanning / CodeQL Workflow does not contain permissions Medium test
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| name: Node.js Production Tests | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - 'plugin/skills/azure-nodejs-production/**' | ||
| - 'tests/nodejs-production/**' | ||
| pull_request: | ||
| branches: [main] | ||
| paths: | ||
| - 'plugin/skills/azure-nodejs-production/**' | ||
| - 'tests/nodejs-production/**' | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| test: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| strategy: | ||
| matrix: | ||
| node-version: [20, 22] | ||
|
|
||
| defaults: | ||
| run: | ||
| working-directory: tests/nodejs-production | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| cache: 'npm' | ||
| cache-dependency-path: tests/nodejs-production/package-lock.json | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci || npm install | ||
|
|
||
| - name: Run tests | ||
| run: npm test | ||
|
|
||
| - name: Run tests with coverage | ||
| run: npm run test:coverage | ||
|
|
||
| - name: Upload coverage to Codecov | ||
| if: matrix.node-version == 20 | ||
| uses: codecov/codecov-action@v4 | ||
| with: | ||
| directory: tests/nodejs-production/coverage | ||
| flags: nodejs-production | ||
| fail_ci_if_error: false | ||
| env: | ||
| CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
Check warningCode scanning / CodeQL Workflow does not contain permissions Medium test
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Line 19 states there are 264 tests in the detection suite, but the PR description and multiple other places in the documentation (line 96, tests/README.md line 96, tests/detection/README.md line 45) state 181 tests. This inconsistency should be corrected to match the actual test count.