Preparing for import defer

[jasnell]

TC-39 is steadily advancing the import defer proposal that will lazily defer the evaluation phase of module loading when possible to do so. V8 has yet to start implementing the proposal so we have some time but there are a few things we will need to figure out both here and in tooling.

As a brief review, let's look at a brief example of what import defer is doing. Let's say my application imports a somewhat expensive dependency...

import { foo } from 'expensive-to-evaluate';

function callFoo() {
  foo();
}

function maybeCallFoo() {
  if (shouldCallFoo()) { callFoo(); }
}

Under the current model, this script will incur the cost of evaluating expensive-to-evaluate whether the foo() function is called or not, leading to larger cold-start times, wasted memory allocations, etc. We can make the import conditional by switching it to a dynamic import...

async function callFoo() {
  const { foo } = await import('expensive-to-evaluate');
  foo();
}

async function maybeCallFoo() {
  if (shouldCallFoo()) { await callFoo(); }
}

... but note that by doing so we are forced to change all of our otherwise synchronous methods to async which carries along a host of problems on its own.

The import defer proposal will allow us to do:

import defer * as thing from 'expensive-to-evaluate';

function callFoo() {
  thing.foo();
}

function maybeCallFoo() {
  if (shouldCallFoo()) { callFoo(); }
}

Such that the expensive-to-evaluate will still be loaded and parsed up front, but the evaluation step will be deferred until the first call to thing.foo()... and herein lies the small challenge for us ... that deferred evaluation occurs synchronously here. Why is that an (somewhat easy to overcome) challenge? Well... in workerd all modules are evaluated outside of the current IoContext. With dynamic import we are able to defer the resolution, parsing, and evaluation asynchronously outside of the current IoContext, but with import defer, if that first call to thing.foo() happens while we are in the scope of an IoContext and we do nothing about it, then that evaluation will happen within the scope of that IoContext, which obviously carries with it a range of issues.

The "simple" solution for us will be to temporarily move the IoContext out of the thread-local storage just before evaluating and restoring it just after. For synthetic modules (CJS, WASM, etc) this is straightforward. For ESM this will possibly require a v8 patch adding a hook that'll tell us when it is about to evaluate the module.

I'm opening this issue to put this on our radar but this is not immediately actionable. When v8 starts working on import defer we will need to pick this up and should try to have it ready by the time that is ready to ship.

As an additional note... we will want to revisit wrangler's bundling strategy in order to take advantage of this. Any workers that get bundled with all dependencies in a single script will obviously not see any advantage to using import defer. Tools like wrangler will need to begin taking more advantage of our multi-module support to see benefit.