chore(doc): added new instructions for e2e test with different lxd ve…

…rsions

Signed-off-by: Mason Hu <[email protected]>

ARCHITECTURE.MD

@@ -46,4 +46,30 @@ A webhook triggers a [demo](https://github.com/canonical/demos.haus) build for a
 
 The [update_demo.yml](.github/workflows/update_demo.yml) workflow updates a branch called `demo` automatically on any merge to main. This keeps at least one demo instance running and always up to date.
 
-A secret is needed to enable HaProxy to forward requests to the LXD API backend. Ask one of the collaborators for the secret.
+A secret is needed to enable HaProxy to forward requests to the LXD API backend. Ask one of the collaborators for the secret.
+
+## E2E test setup for multiple lxd versions
+
+The e2e test suite is run whenever a pull request is created and updated. This is done in the `browser-e2e-test` job from the `PR checks` github [workflow](https://github.com/canonical/lxd-ui/blob/6b574493501264dbac5722b1c2858f4d0020af75/.github/workflows/pr.yaml#L1). The job is run using the github actions matrix strategy so that the same tests can be carried out against different lxd versions and browser types in isolated github runner environments simultaneously. Currently e2e tests are run against the following parameter combinations:
+
+| lxd version  | browser  |
+| -----------  | -------- |
+| 5.0/stable   | chromium |
+| 5.0/stable   | firefox  |
+| latest/edge  | chromium |
+| latest/edge  | firefox  |
+
+For each parameter combination, a playwright project defined in `playwright.config.ts` is selected by name with reference syntax `browser:lxd-version` and executed. The execution flow is depicted in the diagram below:
+
+```mermaid
+    flowchart TD
+    A(pull request created) -->|other workflow jobs| B(browser-e2e-test)
+    B --> C{Github Matrix Runner}
+    C -->|5.0/stable chromium| D[Run chromium:lxd-5.0-stable]
+    C --> E[...]
+    C -->|latest/edge firefox| F[Run firefox:lxd-latest-edge]
+```
+
+The parameter `lxdVersion` set in the configuration for each playwright project is available in every tests' context and may be used to disable a portion of individual tests or skip tests entirely. You can see an example implementation in this [test](https://github.com/canonical/lxd-ui/blob/6b574493501264dbac5722b1c2858f4d0020af75/tests/iso-volumes.spec.ts#L11) where the test is skipped for an older version of lxd.
+
+To add additional lxd versions for e2e testing, you would have to define two projects in `playwright.config.ts` for chromium and firefox browsers, then set the `lxdVersion` parameter accordingly. Then inside [pr.yaml](https://github.com/canonical/lxd-ui/blob/6b574493501264dbac5722b1c2858f4d0020af75/.github/workflows/pr.yaml#L1) include the new lxd versions for the `browser-e2e-test` job. Once these configurations are added, then you can write tests according to your specific use cases for the new lxd versions.

HACKING.md

@@ -102,9 +102,19 @@ Install playwright and its browsers
 
     npx playwright install
 
-The tests expect the environment on localhost to be accessible. Execute `dotrun` and start tests with
+The e2e tests can be run against the LTS version (v5.0) or the latest release version for LXD. If you want to run the tests against the edge version, first make sure your lxd is up to date with
 
-    npx playwright test
+    snap refresh lxd --channel latest/edge
+
+The tests expect the environment on localhost to be accessible. Execute `dotrun` first then run the tests against the latest LXD version with
+
+    yarn test-e2e-edge
+
+or against the LTS LXD version with
+
+    yarn test-e2e-stable
+
+### Nice utilities from Playwright
 
 Generate new tests with helper [Doc](https://playwright.dev/docs/codegen)