From a84c519cced805b09439a34529b1e3302d39bd03 Mon Sep 17 00:00:00 2001 From: Mike Pitre <12040919+mikepitre@users.noreply.github.com> Date: Tue, 3 Nov 2026 16:20:53 -0500 Subject: [PATCH 1/6] docs(verify): open the PR as a draft until the evidence is in CodeRabbit reviews a PR when it opens or turns ready and judges the description it reads then. A PR opened ready was reviewed before attach wrote the evidence, so its device evidence check failed (clerk-ios#805). Co-Authored-By: Claude Opus 5.5 --- .claude/skills/verify-clerk-android/SKILL.md | 4 ++-- .claude/skills/verify-clerk-android/references/remote.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.claude/skills/verify-clerk-android/SKILL.md b/.claude/skills/verify-clerk-android/SKILL.md index a02912756..35e03d105 100644 --- a/.claude/skills/verify-clerk-android/SKILL.md +++ b/.claude/skills/verify-clerk-android/SKILL.md @@ -244,9 +244,9 @@ When this machine cannot attach and there is no session to hand the files to, or Attach the run of your own change. Run your new or changed spec on its own and attach that run, so the screenshots or the video show only what the change is about. A run of `--all` marks nothing, so `attach --video` refuses it. If you ran other committed specs too, cite that run's id in the PR. -Work in this order, with one `up` for both runs: make the branch, `git push` it when the session is remote, `up`, `run` the spec without your change when it changes something that exists, make the change, commit, `git push`, `run` the spec again, open the pull request, `attach` with the before run as `--before `, `down`. Open the pull request after the last run, so that it is not left open without its evidence. Commit before the second `run`, so that the line `attach` writes names a commit the pull request holds. In the pull request, name the run id and the specs you ran, and say what you did not run. +Work in this order, with one `up` for both runs: make the branch, `git push` it when the session is remote, `up`, `run` the spec without your change when it changes something that exists, make the change, commit, `git push`, `run` the spec again, open the pull request as a draft with `gh pr create --draft`, `attach` with the before run as `--before `, `down`, then `gh pr ready ` once the description holds the evidence. Open the pull request after the last run, so that it is not left open without its evidence. Open it as a draft because review bots such as CodeRabbit review a pull request when it opens or turns ready, and judge the description they read then. A pull request opened ready is reviewed before `attach` writes the evidence, and fails the device evidence check. Commit before the second `run`, so that the line `attach` writes names a commit the pull request holds. In the pull request, name the run id and the specs you ran, and say what you did not run. -On a local emulator `attach` writes the description itself, and no run needs a push. A remote session builds only the commit it was started on and later commits of the branch, so its before run comes before the push of the change. Its evidence reaches the description about a minute after `down`. +On a local emulator `attach` writes the description itself, and no run needs a push. A remote session builds only the commit it was started on and later commits of the branch, so its before run comes before the push of the change. Its evidence reaches the description about a minute after `down`, so check that `gh pr view --json body` shows the `verify-evidence` block before `gh pr ready `. When the change is already committed or pushed, make the before run on its base commit in this worktree, so the run lands in this checkout's `.verify/runs/`. A spec in `specs/explored/` stays through the switch, because that directory is gitignored. A committed spec of the change is not on the base, so copy it to `specs/explored/` first and change its import to `'../fixtures.ts'`. diff --git a/.claude/skills/verify-clerk-android/references/remote.md b/.claude/skills/verify-clerk-android/references/remote.md index 5f0941aa4..7e425db7d 100644 --- a/.claude/skills/verify-clerk-android/references/remote.md +++ b/.claude/skills/verify-clerk-android/references/remote.md @@ -105,7 +105,7 @@ The workflow publishes only when all of these hold. Otherwise it prints the reas `attach` checks the same rules before it sends anything, and fails with the reason. -So one session does both runs: push the branch, `up`, `run` for the before run, push the change, `run` again, open the pull request, `attach` with `--before `, `down`. Open the pull request after the last run, so that it is not left open without its evidence, which reaches the description about a minute after `down`. +So one session does both runs: push the branch, `up`, `run` for the before run, push the change, `run` again, open the pull request with `gh pr create --draft`, `attach` with `--before `, `down`, then `gh pr ready ` once the description shows the evidence. Open the pull request after the last run, so that it is not left open without its evidence, which reaches the description about a minute after `down`. It stays a draft until then so that review bots read the description with the evidence in it. A before run is the build without the change, so its commit is often the one the branch started from, which is no commit of the pull request. The workflow therefore does not look for that commit among the commits of the pull request. It asks GitHub whether that commit is the commit of the run or a commit before it in its history, and it publishes nothing when the answer is no. After a rebase that rewrote the commit of the before run, `run` the spec again on a commit that the branch still has, and pass that run as `--before`. The workflow cannot check that the files of the before run came from a build of that commit, which is why the line says that the session reports the before run. From e9589b1e48305bf84fffc325bb0239746a58a452 Mon Sep 17 00:00:00 2001 From: Mike Pitre <12040919+mikepitre@users.noreply.github.com> Date: Tue, 3 Nov 2026 16:30:23 -0500 Subject: [PATCH 2/6] docs(verify): add the draft and ready steps to the remote command sequence Co-Authored-By: Claude Opus 5.5 --- .claude/skills/verify-clerk-android/SKILL.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.claude/skills/verify-clerk-android/SKILL.md b/.claude/skills/verify-clerk-android/SKILL.md index 35e03d105..10817e979 100644 --- a/.claude/skills/verify-clerk-android/SKILL.md +++ b/.claude/skills/verify-clerk-android/SKILL.md @@ -267,8 +267,11 @@ $ e2e-tests/bin/control-clerk-android down $ git switch $ e2e-tests/bin/control-clerk-android up --backend remote $ e2e-tests/bin/control-clerk-android run specs/explored/.e2e.ts +$ gh pr create --draft $ e2e-tests/bin/control-clerk-android attach --pr --screenshot