Playwright: Use new GH Pages actions (#1504)

Co-authored-by: Timur Olzhabayev <[email protected]>

Author: sunker

Diff for: .github/workflows/playwright.yml

@@ -6,8 +6,9 @@ on:
       - '.github/workflows/playwright.yml'
 
 permissions:
-  contents: read
+  contents: write
   id-token: write
+  pull-requests: write
 
 jobs:
   resolve-versions:
@@ -82,9 +83,21 @@ jobs:
         id: run-tests
         run: npm run playwright:test --w @grafana/plugin-e2e
 
-      - name: Publish report to GCS
-        if: ${{ github.repository_owner == 'grafana' && (failure() && steps.run-tests.outcome == 'failure') }}
-        uses: grafana/plugin-actions/publish-report@main
+      - name: Upload e2e test summary
+        uses: grafana/plugin-actions/playwright-gh-pages/upload-report-artifacts@main
+        if: ${{ always() && !cancelled() }}
         with:
-          grafana-version: ${{ matrix.GRAFANA_IMAGE.VERSION }}
-          path: packages/plugin-e2e/playwright-report
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          test-outcome: ${{ steps.run-tests.outcome }}
+          report-dir: packages/plugin-e2e/playwright-report
+
+  publish-report:
+    if: ${{ always() && !cancelled() }}
+    needs: [playwright-tests]
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Publish report
+        uses: grafana/plugin-actions/playwright-gh-pages/deploy-report-pages@main
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}

Diff for: docusaurus/docs/e2e-test-a-plugin/ci.md

@@ -72,8 +72,68 @@ queryString="current-package-manager"
 />
 </details>
 
-## Playwright report
+## Publish Playwright reports to GitHub Pages
 
-The end-to-end tooling generates a Playwright HTML test report for every Grafana version that is being tested. In case any of the tests fail, a Playwright trace viewer is also generated along with the report. The `Upload artifacts` step in the example workflows uploads the report to GitHub as an artifact.
+The Playwright [HTML report](https://playwright.dev/docs/test-reporters#html-reporter), along with the [Trace Viewer](https://playwright.dev/docs/trace-viewer), provides powerful tools for troubleshooting issues found during the execution of end-to-end test. This section explains how to deploy these reports to GitHub's static site hosting service GitHub Pages, making them immediately accessible for review after tests complete.
 
-To find information on how to download and view the report, refer to the [Playwright documentation](https://playwright.dev/docs/ci-intro#html-report).
+This guide is based on the example workflow provided earlier in this document.
+
+### Steps to enable report publishing
+
+1. Immediately following the step that executes the tests, add a step that uses the `upload-report-artifacts` Action to upload the report and a test summary as an to GitHub artifacts.
+
+```yml
+- name: Run Playwright tests
+  id: run-tests
+  run: npx playwright test
+
+- name: Upload e2e test summary
+  uses: grafana/plugin-actions/playwright-gh-pages/upload-report-artifacts@main
+  if: ${{ (always() && !cancelled()) }}
+  with:
+    github-token: ${{ secrets.GITHUB_TOKEN }}
+    test-outcome: ${{ steps.run-tests.outcome }}
+```
+
+2. After the `playwright-tests` job, add a new job to download the report artifacts, deploy them to GitHub Pages, and publish a PR comment summarizing the test results, including links to the reports.
+
+```yml
+publish-report:
+  if: ${{ (always() && !cancelled()) }}
+  needs: [playwright-tests]
+  runs-on: ubuntu-latest
+  steps:
+    - uses: actions/checkout@v4
+    - name: Publish report
+      uses: grafana/plugin-actions/playwright-gh-pages/deploy-report-pages@main
+      with:
+        github-token: ${{ secrets.GITHUB_TOKEN }}
+```
+
+3. Modify the workflow permissions to allow it to push changes, query the GitHub API and update PR comments.
+
+```yml
+permissions:
+  contents: write
+  id-token: write
+  pull-requests: write
+```
+
+4. If GitHub Pages is not yet enabled for your repository, configure a source branch for deployment. Follow the detailed instructions [here](https://github.com/grafana/plugin-actions/tree/main/playwright-gh-pages#github-pages-branch-configuration) to set it up.
+
+For additional configuration options and examples, refer to the `playwright-gh-pages` [documentation](https://github.com/grafana/plugin-actions/blob/main/playwright-gh-pages/README.md).
+
+### Important considerations
+
+- **Public visibility**: By default, GitHub Pages sites are publicly accessible on the Internet. If your end-to-end tests include sensitive data or secrets, be aware of potential exposure risks.
+- **Enterprise access control**: If you have a GitHub Enterprise account, you can configure access controls to restrict visibility. For details, refer to the [GitHub documentation](https://docs.github.com/en/enterprise-cloud@latest/pages/getting-started-with-github-pages/changing-the-visibility-of-your-github-pages-site).
+
+### Report summary
+
+The `publish-report` job adds a PR comment summarizing all the tests executed as part of the matrix. For tests that failed, the comment includes links to the GitHub Pages website, where the detailed reports can be browsed.
+
+![](/img/e2e-report-summary.png)
+
+```
+
+```

Diff for: docusaurus/docs/snippets/plugin-e2e-ds-workflow.npm.md

@@ -68,14 +68,4 @@ jobs:
       - name: Run Playwright tests
         id: run-tests
         run: npx playwright test
-
-      # Uncomment this step to upload the Playwright report to Github artifacts.
-      # If your repository is public, the report will be public on the Internet so beware not to expose sensitive information.
-      # - name: Upload artifacts
-      #   uses: actions/upload-artifact@v4
-      #   if: ${{ (always() && steps.run-tests.outcome == 'success') || (failure() && steps.run-tests.outcome == 'failure') }}
-      #   with:
-      #     name: playwright-report-${{ matrix.GRAFANA_IMAGE.NAME }}-v${{ matrix.GRAFANA_IMAGE.VERSION }}-${{github.run_id}}
-      #     path: playwright-report/
-      #     retention-days: 30
 ```

Diff for: docusaurus/docs/snippets/plugin-e2e-ds-workflow.pnpm.md

@@ -70,14 +70,4 @@ jobs:
       - name: Run Playwright tests
         id: run-tests
         run: pnpm playwright test
-
-      # Uncomment this step to upload the Playwright report to Github artifacts.
-      # If your repository is public, the report will be public on the Internet so beware not to expose sensitive information.
-      # - name: Upload artifacts
-      #   uses: actions/upload-artifact@v4
-      #   if: ${{(always() && steps.run-tests.outcome == 'success') || (failure() && steps.run-tests.outcome == 'failure') }}
-      #   with:
-      #     name: playwright-report-${{ matrix.GRAFANA_IMAGE.NAME }}-v${{ matrix.GRAFANA_IMAGE.VERSION }}-${{github.run_id}}
-      #     path: playwright-report/
-      #     retention-days: 30
 ```

Diff for: docusaurus/docs/snippets/plugin-e2e-ds-workflow.yarn.md

@@ -68,14 +68,4 @@ jobs:
       - name: Run Playwright tests
         id: run-tests
         run: yarn playwright test
-
-      # Uncomment this step to upload the Playwright report to Github artifacts.
-      # If your repository is public, the report will be public on the Internet so beware not to expose sensitive information.
-      # - name: Upload artifacts
-      #   uses: actions/upload-artifact@v4
-      #   if: ${{ (always() && steps.run-tests.outcome == 'success') || (failure() && steps.run-tests.outcome == 'failure') }}
-      #   with:
-      #     name: playwright-report-${{ matrix.GRAFANA_IMAGE.NAME }}-v${{ matrix.GRAFANA_IMAGE.VERSION }}-${{github.run_id}}
-      #     path: playwright-report/
-      #     retention-days: 30
 ```

Diff for: docusaurus/docs/snippets/plugin-e2e-fe-plugin-workflow.npm.md

@@ -60,14 +60,4 @@ jobs:
       - name: Run Playwright tests
         id: run-tests
         run: npx playwright test
-
-      # Uncomment this step to upload the Playwright report to Github artifacts.
-      # If your repository is public, the report will be public on the Internet so beware not to expose sensitive information.
-      # - name: Upload artifacts
-      #   uses: actions/upload-artifact@v4
-      #   if: ${{ (always() && steps.run-tests.outcome == 'success') || (failure() && steps.run-tests.outcome == 'failure') }}
-      #   with:
-      #     name: playwright-report-${{ matrix.GRAFANA_IMAGE.NAME }}-v${{ matrix.GRAFANA_IMAGE.VERSION }}-${{github.run_id}}
-      #     path: playwright-report/
-      #     retention-days: 30
 ```

Diff for: docusaurus/docs/snippets/plugin-e2e-fe-plugin-workflow.pnpm.md

@@ -62,14 +62,4 @@ jobs:
       - name: Run Playwright tests
         id: run-tests
         run: pnpm playwright test
-
-      # Uncomment this step to upload the Playwright report to Github artifacts.
-      # If your repository is public, the report will be public on the Internet so beware not to expose sensitive information.
-      # - name: Upload artifacts
-      #   uses: actions/upload-artifact@v4
-      #   if: ${{ (always() && steps.run-tests.outcome == 'success') || (failure() && steps.run-tests.outcome == 'failure') }}
-      #   with:
-      #     name: playwright-report-${{ matrix.GRAFANA_IMAGE.NAME }}-v${{ matrix.GRAFANA_IMAGE.VERSION }}-${{github.run_id}}
-      #     path: playwright-report/
-      #     retention-days: 30
 ```

Diff for: docusaurus/docs/snippets/plugin-e2e-fe-plugin-workflow.yarn.md

@@ -60,14 +60,4 @@ jobs:
       - name: Run Playwright tests
         id: run-tests
         run: yarn playwright test
-
-      # Uncomment this step to upload the Playwright report to Github artifacts.
-      # If your repository is public, the report will be public on the Internet so beware not to expose sensitive information.
-      # - name: Upload artifacts
-      #   uses: actions/upload-artifact@v4
-      #   if: ${{ (always() && steps.run-tests.outcome == 'success') || (failure() && steps.run-tests.outcome == 'failure') }}
-      #   with:
-      #     name: playwright-report-${{ matrix.GRAFANA_IMAGE.NAME }}-v${{ matrix.GRAFANA_IMAGE.VERSION }}-${{github.run_id}}
-      #     path: playwright-report/
-      #     retention-days: 30
 ```

Diff for: docusaurus/website/static/img/e2e-report-summary.png

@@ -34,12 +34,15 @@
    {{ packageManagerName }} run server
    ```
 
-6. Run the E2E tests (using Cypress)
+6. Run the E2E tests (using Playwright)
 
    ```bash
    # Spins up a Grafana instance first that we tests against
    {{ packageManagerName }} run server
 
+   # If you wish to start a certain Grafana version. If not specified will use latest by default
+   GRAFANA_VERSION=11.3.0 {{ packageManagerName }} run server
+
    # Starts the tests
    {{ packageManagerName }} run e2e
    ```

Diff for: packages/create-plugin/templates/_partials/frontend-getting-started.md

@@ -11,7 +11,10 @@ on:
       - master
       - main
 
-permissions: read-all
+permissions:
+  contents: write
+  id-token: write
+  pull-requests: write
 
 jobs:
   build:
@@ -203,6 +206,14 @@ jobs:
         id: run-tests
         run: {{ packageManagerName }} run e2e
 
+      - name: Upload e2e test summary
+        uses: grafana/plugin-actions/playwright-gh-pages/upload-report-artifacts@main
+        if: $\{{ always() && !cancelled() }}
+        with:
+          upload-report: false
+          github-token: $\{{ secrets.GITHUB_TOKEN }}
+          test-outcome: $\{{ steps.run-tests.outcome }}
+
       - name: Docker logs
         if: $\{{ always() && steps.run-tests.outcome == 'failure' }}
         run: |
@@ -211,20 +222,23 @@ jobs:
       - name: Stop grafana docker
         run: docker compose down
 
-      - name: Upload server log
-        uses: actions/upload-artifact@v4
-        if: $\{{ always() && steps.run-tests.outcome == 'failure' }}
-        with:
-          name: $\{{ matrix.GRAFANA_IMAGE.NAME }}-v$\{{ matrix.GRAFANA_IMAGE.VERSION }}-$\{{github.run_id}}-server-log
-          path: grafana-server.log
-          retention-days: 5
-
-      # Uncomment this step to upload the Playwright report to Github artifacts.
-      # If your repository is public, the report will be public on the Internet so beware not to expose sensitive information.
-      # - name: Upload artifacts
+      # Uncomment this step to upload the server log to Github artifacts. Remember Github artifacts are public on the Internet if the repository is public.
+      # - name: Upload server log
       #   uses: actions/upload-artifact@v4
       #   if: $\{{ always() && steps.run-tests.outcome == 'failure' }}
       #   with:
-      #     name: playwright-report-$\{{ matrix.GRAFANA_IMAGE.NAME }}-v$\{{ matrix.GRAFANA_IMAGE.VERSION }}-$\{{github.run_id}}
-      #     path: playwright-report/
+      #     name: $\{{ matrix.GRAFANA_IMAGE.NAME }}-v$\{{ matrix.GRAFANA_IMAGE.VERSION }}-$\{{github.run_id}}-server-log
+      #     path: grafana-server.log
       #     retention-days: 5
+
+  publish-report:
+    if: $\{{ always() && !cancelled() }}
+    needs: [playwright-tests]
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Publish report
+        uses: grafana/plugin-actions/playwright-gh-pages/deploy-report-pages@main
+        with:
+          github-token: $\{{ secrets.GITHUB_TOKEN }}
+