Skip to content
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e485077
Preserve Roadmap display options
SIkebe Aug 25, 2026
04c909a
Remove obsolete compatibility paths
SIkebe Aug 25, 2026
2a4251f
Harden Roadmap display validation
SIkebe Aug 25, 2026
5e76dcf
Make Roadmap capture fail independently
SIkebe Aug 25, 2026
fb328d2
Validate required schema-v2 template state
SIkebe Aug 25, 2026
58de71b
Exercise rendered Roadmap output in shared E2E
SIkebe Aug 25, 2026
8892134
Add mixed-state Roadmap fixture coverage
SIkebe Aug 25, 2026
4e45bdb
Centralize Roadmap rendering selectors
SIkebe Aug 25, 2026
df3bd58
Count only visible Roadmap dates
SIkebe Aug 25, 2026
e42898a
Model Roadmap display options as shared state
SIkebe Aug 25, 2026
b2be781
Revert "Harden Roadmap display validation"
SIkebe Aug 25, 2026
52e07fc
Revert "Preserve Roadmap display options"
SIkebe Aug 25, 2026
e58b1fc
Document unsupported Roadmap display controls
SIkebe Aug 25, 2026
04e88ef
Reject incomplete schema-v2 snapshots
SIkebe Aug 25, 2026
6420d8d
Normalize malformed snapshot errors
SIkebe Aug 25, 2026
ca885b2
Require explicit schema-v2 state
SIkebe Aug 25, 2026
29a938f
Align schema-v2 documentation
SIkebe Aug 25, 2026
ebe404a
Reapply "Preserve Roadmap display options"
SIkebe Aug 26, 2026
974ee0e
Reapply "Harden Roadmap display validation"
SIkebe Aug 26, 2026
23a6953
Persist browser-only view settings across sessions
SIkebe Aug 26, 2026
be72928
Flush recoverable Roadmap writes
SIkebe Aug 26, 2026
357ab38
Keep profile persistence regression lightweight
SIkebe Aug 26, 2026
6afd0c1
Verify Roadmap display controls independently
SIkebe Aug 26, 2026
dbb2ec6
Validate Roadmap state before import writes
SIkebe Aug 26, 2026
591168a
Target Roadmap pills in rendering checks
SIkebe Aug 26, 2026
1a05ea2
Restore browser profile after E2E drift
SIkebe Aug 26, 2026
d5c225c
Describe all retained E2E drifts
SIkebe Aug 26, 2026
2b3a106
Test partial Roadmap state persistence
SIkebe Aug 26, 2026
cbd23e6
Gate Roadmap rendering fixture before import
SIkebe Aug 26, 2026
93dbfd7
Retry delayed field default persistence
SIkebe Aug 27, 2026
89d6fb8
Explain live API failures on pull requests
SIkebe Aug 28, 2026
de680e4
Isolate live API failure reporting
SIkebe Aug 28, 2026
b8e8df6
Fix trusted reporting and fixture guidance
SIkebe Aug 28, 2026
6e12dd7
Sanitize integration test names in PR comments
SIkebe Aug 28, 2026
673c6f5
Keep reusable live API workflow read-only
SIkebe Aug 28, 2026
af78fe7
Scope reporter token to individual steps
SIkebe Aug 28, 2026
c9f39af
Warn before using undocumented Roadmap UI
SIkebe Aug 28, 2026
5c58cee
Isolate live API comments by commit
SIkebe Aug 28, 2026
6c8d5ed
Serialize live API status comments per PR
SIkebe Aug 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
263 changes: 248 additions & 15 deletions .github/skills/ghpmv-e2e-validation/SKILL.md

Large diffs are not rendered by default.

176 changes: 176 additions & 0 deletions .github/workflows/live-api-report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
name: Report Live GitHub API

on:
workflow_run:
workflows: [CI]
types: [completed]

permissions:
actions: read
contents: read
pull-requests: write

defaults:
run:
shell: pwsh

jobs:
Comment thread
SIkebe marked this conversation as resolved.
report:
name: Report integration result
if: ${{ github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.pull_requests[0] != null }}
runs-on: ubuntu-latest
timeout-minutes: 5
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.workflow_run.pull_requests[0].number }}
REPOSITORY: ${{ github.repository }}
RUN_ID: ${{ github.event.workflow_run.id }}
RUN_URL: ${{ github.event.workflow_run.html_url }}
steps:
- name: Find Live GitHub API result
id: result
run: |
$jobsJson = gh api --paginate --slurp "repos/$env:REPOSITORY/actions/runs/$env:RUN_ID/jobs?per_page=100"
if ($LASTEXITCODE -ne 0) {
throw 'Could not read jobs for the completed CI run.'
}
$jobs = @($jobsJson | ConvertFrom-Json | ForEach-Object { $_.jobs })
$integration = @($jobs | Where-Object name -eq 'Live GitHub API / Integration' | Select-Object -Last 1)
if ($integration.Count -eq 0) {
'result=skipped' | Out-File -FilePath $env:GITHUB_OUTPUT -Encoding utf8 -Append
exit 0
}
$summary = @($jobs | Where-Object name -eq 'Live GitHub API / Summarize integration result' | Select-Object -Last 1)
$result = if ($integration[0].conclusion -eq 'success' -and
($summary.Count -eq 0 -or $summary[0].conclusion -ne 'success')) {
'skipped'
}
else {
$integration[0].conclusion
}
"result=$result" | Out-File -FilePath $env:GITHUB_OUTPUT -Encoding utf8 -Append
- name: Download integration test results
if: ${{ steps.result.outputs.result == 'failure' }}
continue-on-error: true
run: gh run download "$env:RUN_ID" --repo "$env:REPOSITORY" --name live-api-test-results --dir tests/Ghpmv.Integration.Tests/TestResults
- name: Report integration failure
if: ${{ steps.result.outputs.result == 'failure' }}
run: |
$marker = '<!-- ghpmv-live-api-failure -->'
$reportPath = Get-ChildItem -LiteralPath 'tests/Ghpmv.Integration.Tests/TestResults' -Filter 'integration.trx' -File -Recurse -ErrorAction SilentlyContinue |
Select-Object -First 1 -ExpandProperty FullName
$failedTests = @()
$messages = @()
if ($null -ne $reportPath) {
$settings = [System.Xml.XmlReaderSettings]::new()
$settings.DtdProcessing = [System.Xml.DtdProcessing]::Prohibit
$settings.XmlResolver = $null
$reader = [System.Xml.XmlReader]::Create($reportPath, $settings)
$results = [System.Xml.XmlDocument]::new()
$results.XmlResolver = $null
try {
$results.Load($reader)
}
finally {
$reader.Dispose()
}
$failures = @($results.SelectNodes("//*[local-name()='UnitTestResult' and @outcome='Failed']"))
foreach ($failure in $failures) {
$failedTests += $failure.testName
$message = $failure.SelectSingleNode("./*[local-name()='Output']/*[local-name()='ErrorInfo']/*[local-name()='Message']")
if ($null -ne $message) {
$messages += $message.InnerText
}
}
}

$failureText = $messages -join "`n"
$diagnosis = if ($failureText -match 'Can only add 1000 curated projects to a repo') {
"The target fixture repository reached GitHub's 1,000 linked-Projects limit. Rotate the target fixture repository; rerunning without replacing it will fail again."
}
elseif ($failureText -match 'could not link repository') {
"Repository linking failed. Confirm the target fixture exists, the test token can administer it, and the repository has not reached GitHub's linked-Projects limit."
}
elseif ($failureText -match '(?i)rate limit') {
'GitHub API rate limiting interrupted the live test. Review the rate-limit entries in the run log before rerunning.'
}
elseif ($failureText -match 'Resource not accessible by personal access token') {
'The test token lacks access required by the failing GitHub operation. Review its scopes, repository access, approval, and SSO authorization.'
}
else {
'The live GitHub API test failed for an unclassified reason. Inspect the run log and uploaded TRX artifact.'
}

$testLines = if ($failedTests.Count -gt 0) {
@($failedTests | Sort-Object -Unique | ForEach-Object {
$safeName = [System.Net.WebUtility]::HtmlEncode($_).Replace('`', '&#96;')
"- ``$safeName``"
})
Comment thread
SIkebe marked this conversation as resolved.
}
else {
@('- Failed test names were unavailable; inspect the run log.')
}
$body = (@(
$marker
'### Live GitHub API check failed'
''
"**Diagnosis:** $diagnosis"
''
'**Failed tests:**'
) + $testLines + @(
''
"[Open the Actions run]($env:RUN_URL)"
)) -join "`n"

Add-Content -LiteralPath $env:GITHUB_STEP_SUMMARY -Value ($body.Replace($marker, '').Trim())
Write-Output "::error title=Live GitHub API check failed::$diagnosis"
$commentsJson = gh api --paginate --slurp "repos/$env:REPOSITORY/issues/$env:PR_NUMBER/comments?per_page=100"
if ($LASTEXITCODE -ne 0) {
Write-Output '::warning title=PR failure comment was not posted::Could not list existing pull request comments.'
exit 0
}
$comments = @($commentsJson | ConvertFrom-Json | ForEach-Object { $_ })
$comment = @($comments | Where-Object {
$_.user.login -eq 'github-actions[bot]' -and $_.body.Contains($marker, [StringComparison]::Ordinal)
} | Select-Object -Last 1)
$payload = @{ body = $body } | ConvertTo-Json -Compress
if ($comment.Count -eq 1) {
$payload | gh api --method PATCH "repos/$env:REPOSITORY/issues/comments/$($comment[0].id)" --input -
}
else {
$payload | gh api --method POST "repos/$env:REPOSITORY/issues/$env:PR_NUMBER/comments" --input -
}
if ($LASTEXITCODE -ne 0) {
Write-Output '::warning title=PR failure comment was not posted::The pull request comment API request failed.'
exit 0
}
- name: Resolve integration failure comment
if: ${{ steps.result.outputs.result == 'success' }}
run: |
$marker = '<!-- ghpmv-live-api-failure -->'
$commentsJson = gh api --paginate --slurp "repos/$env:REPOSITORY/issues/$env:PR_NUMBER/comments?per_page=100"
if ($LASTEXITCODE -ne 0) {
Write-Output '::warning title=PR failure comment was not resolved::Could not list existing pull request comments.'
exit 0
}
$comments = @($commentsJson | ConvertFrom-Json | ForEach-Object { $_ })
$comment = @($comments | Where-Object {
$_.user.login -eq 'github-actions[bot]' -and $_.body.Contains($marker, [StringComparison]::Ordinal)
} | Select-Object -Last 1)
if ($comment.Count -eq 0) {
Write-Output 'No prior Live GitHub API failure comment exists.'
exit 0
}

$body = @"
$marker
### Live GitHub API check recovered

The Live GitHub API check passed on the [latest Actions run]($env:RUN_URL).
"@
$payload = @{ body = $body.Trim() } | ConvertTo-Json -Compress
$payload | gh api --method PATCH "repos/$env:REPOSITORY/issues/comments/$($comment[0].id)" --input -
if ($LASTEXITCODE -ne 0) {
Write-Output '::warning title=PR failure comment was not resolved::The pull request comment API request failed.'
exit 0
}
80 changes: 80 additions & 0 deletions .github/workflows/live-api.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ on:

permissions:
contents: read
pull-requests: write

concurrency:
group: live-github-api-shared-fixture
Expand All @@ -30,6 +31,8 @@ jobs:
timeout-minutes: 20
permissions:
contents: read
outputs:
credentials-available: ${{ steps.credentials.outputs.available }}
env:
GHPMV_TEST_ORG: ${{ vars.GHPMV_TEST_ORG || 'gpm-source' }}
GHPMV_TEST_PROJECT_NUMBER: ${{ vars.GHPMV_TEST_PROJECT_NUMBER || '89' }}
Expand Down Expand Up @@ -80,6 +83,7 @@ jobs:
if: ${{ steps.credentials.outputs.available == 'true' }}
run: dotnet build Ghpmv.slnx -c Release --no-restore -warnaserror
- name: Synchronize fixture and test
id: integration
if: ${{ steps.credentials.outputs.available == 'true' }}
env:
GHPMV_TEST_TOKEN: ${{ secrets.GHPMV_TEST_TOKEN }}
Expand Down Expand Up @@ -111,3 +115,79 @@ jobs:
path: tests/Ghpmv.Integration.Tests/TestResults/**
if-no-files-found: warn
retention-days: 7

report:
name: Summarize integration result
needs: integration
if: ${{ always() && needs.integration.outputs.credentials-available == 'true' }}
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Download integration test results
if: ${{ needs.integration.result == 'failure' }}
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: live-api-test-results
path: tests/Ghpmv.Integration.Tests/TestResults
- name: Report integration failure
if: ${{ needs.integration.result == 'failure' }}
env:
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
$marker = '<!-- ghpmv-live-api-failure -->'
$reportPath = Get-ChildItem -LiteralPath 'tests/Ghpmv.Integration.Tests/TestResults' -Filter 'integration.trx' -File -Recurse -ErrorAction SilentlyContinue |
Select-Object -First 1 -ExpandProperty FullName
$failedTests = @()
$messages = @()
if ($null -ne $reportPath) {
[xml]$results = Get-Content -LiteralPath $reportPath
$failures = @($results.SelectNodes("//*[local-name()='UnitTestResult' and @outcome='Failed']"))
foreach ($failure in $failures) {
$failedTests += $failure.testName
$message = $failure.SelectSingleNode("./*[local-name()='Output']/*[local-name()='ErrorInfo']/*[local-name()='Message']")
if ($null -ne $message) {
$messages += $message.InnerText
}
}
}

$failureText = $messages -join "`n"
$diagnosis = if ($failureText -match 'Can only add 1000 curated projects to a repo') {
"The target fixture repository reached GitHub's 1,000 linked-Projects limit. Rotate the target fixture repository; rerunning without replacing it will fail again."
}
elseif ($failureText -match 'could not link repository') {
"Repository linking failed. Confirm the target fixture exists, the test token can administer it, and the repository has not reached GitHub's linked-Projects limit."
}
elseif ($failureText -match '(?i)rate limit') {
'GitHub API rate limiting interrupted the live test. Review the rate-limit entries in the run log before rerunning.'
}
elseif ($failureText -match 'Resource not accessible by personal access token') {
'The test token lacks access required by the failing GitHub operation. Review its scopes, repository access, approval, and SSO authorization.'
}
else {
'The live GitHub API test failed for an unclassified reason. Inspect the run log and uploaded TRX artifact.'
}

$testLines = if ($failedTests.Count -gt 0) {
@($failedTests | Sort-Object -Unique | ForEach-Object { "- ``$_``" })
}
else {
@('- Failed test names were unavailable; inspect the run log.')
}
$body = (@(
$marker
'### Live GitHub API check failed'
''
"**Diagnosis:** $diagnosis"
''
'**Failed tests:**'
) + $testLines + @(
''
"[Open the Actions run]($env:RUN_URL)"
)) -join "`n"

Add-Content -LiteralPath $env:GITHUB_STEP_SUMMARY -Value ($body.Replace($marker, '').Trim())
Write-Output "::error title=Live GitHub API check failed::$diagnosis"
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Tokens are resolved from `--token`, then the `GITHUB_TOKEN` / `GHPMV_TOKEN` envi

| Category | Verification coverage |
|---|---|
| Project | Description, README, visibility, closed state, and organization template state. A changed title is informational because import supports title overrides. Legacy snapshots without `project.template` preserve and do not compare the target template state. |
| Project | Description, README, visibility, closed state, and organization template state. A changed title is informational because import supports title overrides. |
| Field | Field presence/type, select option order/name/color/description, Issue Field description/visibility/linkage, and iteration dates/durations. |
| Item | Counts/types, issue and pull request identity, draft body, field values (including Project and Issue Field multi-select values), active-item order, and archived state. Archived-item order is excluded because GitHub cannot restore it. |
| StatusUpdate | History order, body (including the imported attribution note), status, start date, and target date. |
Expand Down
5 changes: 3 additions & 2 deletions docs/BROWSER_AUTOMATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ GraphQL と Playwright を組み合わせた View・Workflow 移行の詳細設
| sort(複数キー+方向) | GraphQL `sortByFields`(`ProjectV2SortByField.direction`) | **UI** | |
| **Slice by** | ❌ API に無い → **UI で読む** | **UI** | |
| **Field sum** | ❌ API に無い → **UI で読む** | **UI** | Board と grouped Table / Roadmap。Count、複数 Number field、空集合を complete-set 同期 |
| **Roadmap 設定(Dates / Zoom / Markers)** | ❌ API に無い → **UI で読む** | **UI** | |
| タブの並び順 | **UI**(`navigation "Select view"` 内のsaved tab `href`順) | **UI**(タブの drag & drop) | GraphQL `POSITION`は現行UIのsaved-tab順と乖離する場合がある。`ViewSnapshot.tabPosition`はschema v1のnullable additive field |
| **Roadmap 設定(Dates / Zoom / Markers / Truncate titles / Show date fields)** | ❌ API に無い → **UI で読む** | **UI** | 表示optionは全Roadmap Viewで共有され、browser storageに保持されるため、各Viewから同一stateをcapture/replayし、書き込み後にprofileを保存 |
| タブの並び順 | **UI**(`navigation "Select view"` 内のsaved tab `href`順) | **UI**(タブの drag & drop) | GraphQL `POSITION`は現行UIのsaved-tab順と乖離する場合がある |

### Workflow のプロパティ別ソースマップ

Expand Down Expand Up @@ -336,6 +336,7 @@ browser importer 自体は各 view / workflow の適用直後に完全な read-b
2. "Fixture Board" — Board, Column by, swimlane, Field sum=[Fixture Number]
3. "Fixture Roadmap" — grouped Roadmap, Field sum=[Fixture Number 2], Dates, Zoom, Markers
4. "Fixture Empty Sums" — grouped Table, Field sum=[]
5. "Fixture Roadmap Dates Hidden" — grouped Roadmap, Truncate titles=on, Show date fields=off
- Workflows: W-1〜W-8 を非デフォルト Status 値で有効化、W-9 を 2 本(別リポ + 別フィルター)。1 つは disabled のまま設定を持たせる(§4.3 の D0 論点の検証用)
- Items: issue 10 / PR 3 / draft 3(archived 2 を含む)
- Field defaults: Fixture Text=`既定値 🌏`、Fixture Number=`-7`、Fixture Number 2=`0`、Fixture Select=`Beta`
Expand Down
Loading