[WPT] Introduce RemoteContext.execute_script() and add basic BFCach…

…e tests + helpers

This PR adds `RemoteContext.execute_script()` and its documentation
in `/common/dispatcher/`.
This is based on with `execute_script()`-related parts of RFCs 88/89/91:

- web-platform-tests/rfcs#88
- web-platform-tests/rfcs#89
- web-platform-tests/rfcs#91

plus additional clarifications around navigation,
minus `testdriver` integration (so this is implemented using
`send()`/`receive()` in `/common/dispatcher/`).

This PR also adds back-forward cache WPTs (basic event firing tests),
as well as BFCache-specific helpers, based on
`RemoteContext.execute_script()`.

Design doc:
https://docs.google.com/document/d/1p3G-qNYMTHf5LU9hykaXcYtJ0k3wYOwcdVKGeps6EkU/edit?usp=sharing

(Note: this CL is tentatively rebased on `main`, to trigger Blink WPT bot,
as this CL depends on
https://chromium-review.googlesource.com/c/chromium/src/+/3033199.
After CL 3033199 lands, this CL should be rebased)

Bug: 1107415
Change-Id: I034f9f5376dc3f9f32ca0b936dbd06e458c9160b

common/dispatcher/README.md

@@ -0,0 +1,210 @@
+# `RemoteContext`: API for script execution in another context
+
+`RemoteContext` in `/common/dispatcher/dispatcher.js` provides an interface to
+execute JavaScript in another global object (page or worker, the "executor"),
+based on
+
+- [WPT RFC 88: context IDs from uuid searchParams in URL](https://github.com/web-platform-tests/rfcs/pull/88),
+- [WPT RFC 89: execute_script](https://github.com/web-platform-tests/rfcs/pull/89) and
+- [WPT RFC 91: RemoteContext](https://github.com/web-platform-tests/rfcs/pull/91).
+
+Tests can send arbitrary javascript to executors to evaluate in its global
+object, like:
+
+```
+// injector.html
+const argOnLocalContext = ...;
+
+async function execute() {
+  window.open('executor.html?uuid=' + uuid);
+  const ctx = new RemoteContext(uuid);
+  await ctx.execute_script(
+      (arg) => functionOnRemoteContext(arg),
+      [argOnLocalContext]);
+};
+```
+
+and on executor:
+
+```
+// executor.html
+function functionOnRemoteContext(arg) { ... }
+
+const uuid = new URLSearchParams(window.location.search).get('uuid');
+const executor = new Executor(uuid);
+```
+
+For concrete examples, see
+[events.html](../../html/browsers/browsing-the-web/back-forward-cache/events.html)
+and
+[executor.html](../../html/browsers/browsing-the-web/back-forward-cache/resources/executor.html)
+in back-forward cache tests.
+Note that executor files under `/common/dispatcher/` are NOT for
+`RemoteContext.execute_script()`.
+
+This is universal and avoids introducing many specific `XXX-helper.html`
+resources.
+Moreover, tests are easier to read, because the whole logic of the test can be
+defined in a single file.
+
+## `new RemoteContext(uuid)`
+
+- `uuid` is a UUID string that identifies the remote context and should match
+  with the `uuid` parameter of the URL of the remote context.
+- Callers should create the remote context outside this constructor (e.g.
+  `window.open('executor.html?uuid=' + uuid)`).
+
+## `RemoteContext.execute_script(fn, args)`
+
+- `fn` is a JavaScript function to execute on the remote context, which is
+  converted to a string using `toString()` and sent to the remote context.
+- `args` is null or an array of arguments to pass to the function on the
+  remote context. Arguments are passed as JSON.
+- If the return value of `fn` when executed in the remote context is a promise,
+  the promise returned by `execute_script` resolves to the resolved value of
+  that promise. Otherwise the `execute_script` promise resolves to the return
+  value of `fn`.
+
+Note that `fn` is evaluated on the remote context (`executor.html` in the
+example above), while `args` are evaluated on the caller context
+(`injector.html`) and then passed to the remote context.
+
+## Return value of injected functions and `execute_script()`
+
+If the return value of the injected function when executed in the remote
+context is a promise, the promise returned by `execute_script` resolves to the
+resolved value of that promise. Otherwise the `execute_script` promise resolves
+to the return value of the function.
+
+When the return value of an injected script is a Promise, it should be resolved
+before any navigation starts on the remote context. For example, it shouldn't
+be resolved after navigating out and navigating back to the page again.
+It's fine to create a Promise to be resolved after navigations, if it's not the
+return value of the injected function.
+
+## Calling timing of `execute_script()`
+
+When `RemoteContext.execute_script()` is called when the remote context is not
+active (for example before it is created, before navigation to the page, or
+during the page is in back-forward cache), the injected script is evaluated
+after the remote context becomes active.
+
+`RemoteContext.execute_script()` calls should be serialized by always waiting
+for the returned promise to be resolved.
+So it's a good practice to always write `await ctx.execute_script(...)`.
+
+## Evaluation timing of injected functions
+
+The script injected by `RemoteContext.execute_script()` can be evaluated any
+time during the remote context is active.
+For example, even before DOMContentLoaded events or even during navigation.
+It's the responsibility of test-specific code/helpers to ensure evaluation
+timing constraints (which can be also test-specific), if any needed.
+
+### Ensuring evaluation timing around page load
+
+For example, to ensure that injected functions (`mainFunction` below) are
+evaluated after the first `pageshow` event, we can use pure JavaScript code
+like below:
+
+```
+// executor.html
+window.pageShowPromise = new Promise(resolve =>
+    window.addEventListener('pageshow', resolve, {once: true}));
+
+
+// injector.html
+const waitForPageShow = async () => {
+  while (!window.pageShowPromise) {
+    await new Promise(resolve => setTimeout(resolve, 100));
+  }
+  await window.pageShowPromise;
+};
+
+await ctx.execute(waitForPageShow);
+await ctx.execute(mainFunction);
+```
+
+### Ensuring evaluation timing around navigation out/unloading
+
+It can be important to ensure there are no injected functions nor code behind
+`RemoteContext` (such as Fetch APIs accessing server-side stash) running after
+navigation is initiated, for example in the case of back-forward cache testing.
+
+To ensure this,
+
+- Do not call the next `RemoteContext.execute()` for the remote context after
+  triggering the navigation, until we are sure that the remote context is not
+  active (e.g. after we confirm that the new page is loaded).
+- Call `Executor.suspend(callback)` synchronously within the injected script.
+  This suspends executor-related code, and calls `callback` when it is ready
+  to start navigation.
+
+The code on the injector side would be like:
+
+```
+// injector.html
+await ctx.execute_script(() => {
+  executor.suspend(() => {
+    location.href = 'new-url.html';
+  });
+});
+```
+
+## Future Work: Possible integration with `test_driver`
+
+Currently `RemoteContext` is implemented by JavaScript and WPT-server-side
+stash, and not integrated with `test_driver` nor `testharness`.
+There is a proposal of `test_driver`-integrated version (see the RFCs listed
+above).
+
+The API semantics and guidelines in this document are designed to be applicable
+to both the current stash-based `RemoteContext` and `test_driver`-based
+version, and thus the tests using `RemoteContext` will be migrated with minimum
+modifications (mostly in `/common/dispatcher/dispatcher.js` and executors), for
+example in a
+[draft CL](https://chromium-review.googlesource.com/c/chromium/src/+/3082215/).
+
+
+# `send()`/`receive()` Message passing APIs
+
+`dispatcher.js` (and its server-side backend `dispatcher.py`) provides a
+universal queue-based message passing API.
+Each queue is identified by a UUID, and accessed via the following APIs:
+
+-   `send(uuid, message)` pushes a string `message` to the queue `uuid`.
+-   `receive(uuid)` pops the first item from the queue `uuid`.
+-   `showRequestHeaders(origin, uuid)` and
+    `cacheableShowRequestHeaders(origin, uuid)` return URLs, that push request
+    headers to the queue `uuid` upon fetching.
+
+It works cross-origin, and even access different browser context groups.
+
+Messages are queued, this means one doesn't need to wait for the receiver to
+listen, before sending the first message
+(but still need to wait for the resolution of the promise returned by `send()`
+to ensure the order between `send()`s).
+
+## Executors
+
+Similar to `RemoteContext.execute_script()`, `send()`/`receive()` can be used
+for sending arbitrary javascript to be evaluated in another page or worker.
+
+- `executor.html` (as a Document),
+- `executor-worker.js` (as a Web Worker), and
+- `executor-service-worker.js` (as a Service Worker)
+
+are examples of executors.
+Note that these executors are NOT compatible with
+`RemoteContext.execute_script()`.
+
+## Future Work
+
+`send()`, `receive()` and the executors below are kept for COEP/COOP tests.
+
+For remote script execution, new tests should use
+`RemoteContext.execute_script()` instead.
+
+For message passing,
+[WPT RFC 90](https://github.com/web-platform-tests/rfcs/pull/90) is still under
+discussion.

common/dispatcher/dispatcher.js

@@ -0,0 +1,178 @@
+// Define an universal message passing API. It works cross-origin and across
+// browsing context groups.
+const dispatcher_path = "/common/dispatcher/dispatcher.py";
+const dispatcher_url = new URL(dispatcher_path, location.href).href;
+
+// Return a promise, limiting the number of concurrent accesses to a shared
+// resources to |max_concurrent_access|.
+const concurrencyLimiter = (max_concurrency) => {
+  let pending = 0;
+  let waiting = [];
+  return async (task) => {
+    pending++;
+    if (pending > max_concurrency)
+      await new Promise(resolve => waiting.push(resolve));
+    let result = await task();
+    pending--;
+    waiting.shift()?.();
+    return result;
+  };
+}
+
+// Wait for a random amount of time in the range [10ms,100ms].
+const randomDelay = () => {
+  return new Promise(resolve => setTimeout(resolve, 10 + 90*Math.random()));
+}
+
+// Sending too many requests in parallel causes congestion. Limiting it improves
+// throughput.
+//
+// Note: The following table has been determined on the test:
+// ../cache-storage.tentative.https.html
+// using Chrome with a 64 core CPU / 64GB ram, in release mode:
+// ┌───────────┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬────┐
+// │concurrency│ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 10│ 15│ 20│ 30│ 50│ 100│
+// ├───────────┼───┼───┼───┼───┼───┼───┼───┼───┼───┼───┼───┼────┤
+// │time (s)   │ 54│ 38│ 31│ 29│ 26│ 24│ 22│ 22│ 22│ 22│ 34│ 36 │
+// └───────────┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴────┘
+const limiter = concurrencyLimiter(6);
+
+const send = async function(uuid, message) {
+  await limiter(async () => {
+    // Requests might be dropped. Retry until getting a confirmation it has been
+    // processed.
+    while(1) {
+      try {
+        let response = await fetch(dispatcher_url + `?uuid=${uuid}`, {
+          method: 'POST',
+          body: message
+        })
+        if (await response.text() == "done")
+          return;
+      } catch (fetch_error) {}
+      await randomDelay();
+    };
+  });
+}
+
+const receive = async function(uuid) {
+  while(1) {
+    let data = "not ready";
+    try {
+      data = await limiter(async () => {
+        let response = await fetch(dispatcher_url + `?uuid=${uuid}`);
+        return await response.text();
+      });
+    } catch (fetch_error) {}
+
+    if (data == "not ready") {
+      await randomDelay();
+      continue;
+    }
+
+    return data;
+  }
+}
+
+// Returns an URL. When called, the server sends toward the `uuid` queue the
+// request headers. Useful for determining if something was requested with
+// Cookies.
+const showRequestHeaders = function(origin, uuid) {
+  return origin + dispatcher_path + `?uuid=${uuid}&show-headers`;
+}
+
+// Same as above, except for the response is cacheable.
+const cacheableShowRequestHeaders = function(origin, uuid) {
+  return origin + dispatcher_path + `?uuid=${uuid}&cacheable&show-headers`;
+}
+
+// This script requires
+// - `/common/utils.js` for `token()`.
+
+// Represents an remote executor with ID `this.uuid`.
+// For more detailed explanation see `README.md`.
+class RemoteContext {
+  // Caller should create an executor with `this.uuid`.
+  constructor(uuid) {
+    this.context_id = uuid;
+  }
+
+  // Evaluates the script `expr` on the executor.
+  // - If `expr` is evaluated to a Promise that is resolved with a value:
+  //   `eval()` returns a Promise resolved with the value.
+  // - If `expr` is evaluated to a non-Promise value:
+  //   `eval()` returns a Promise resolved with the value.
+  // - If `expr` throws an error or is evaluated to a Promise that is rejected:
+  //   `eval()` returns a rejected Promise with the error's `message`.
+  //   Note that currently the type of error (e.g. DOMException) is not
+  //   preserved.
+  // The values should be able to be serialized by JSON.stringify().
+  async execute_script(fn, args) {
+    const receiver = token();
+    await this.send({receiver: receiver, fn: fn.toString(), args: args});
+    const response = JSON.parse(await receive(receiver));
+    if (response.status === 'success') {
+      return response.value;
+    }
+
+    // exception
+    throw new Error(response.value);
+  }
+
+  async send(msg) {
+    return await send(this.context_id, JSON.stringify(msg));
+  }
+};
+
+class Executor {
+  constructor(uuid) {
+    this.uuid = uuid;
+
+    // If `suspend_callback` is not `null`, the executor should be suspended
+    // when there are no ongoing tasks.
+    this.suspend_callback = null;
+
+    this.execute();
+  }
+
+  suspend(callback) {
+    this.suspend_callback = callback;
+  }
+
+  resume() {
+  }
+
+  async execute() {
+    while(true) {
+      // At this point there are no ongoing tasks and thus it's safe to start
+      // navigation.
+      // Therefore we check whether the executor should be suspended.
+      if (this.suspend_callback !== null) {
+        this.suspend_callback();
+        this.suspend_callback = null;
+        // Wait for `resume()` to be called.
+        await new Promise(resolve => this.resume = resolve);
+        // Workaround for crbug.com/1244230.
+        await new Promise(resolve => setTimeout(resolve, 0));
+        continue;
+      }
+
+      const task = JSON.parse(await receive(this.uuid));
+
+      let response;
+      try {
+        const value = await eval(task.fn).apply(null, task.args);
+        response = JSON.stringify({
+          status: 'success',
+          value: value
+        });
+      } catch(e) {
+        response = JSON.stringify({
+          status: 'exception',
+          value: e.message
+        });
+      }
+      await send(task.receiver, response);
+    }
+  }
+}