Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: chunk importmap #16552

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/config/build-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,3 +279,10 @@ There are cases that file system watching does not work with WSL2.
See [`server.watch`](./server-options.md#server-watch) for more details.

:::

## build.chunkImportMap

- **Type:** `boolean`
- **Default:** `false`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Default:** `false`
- **Default:** `false`
- **Experimental**

#16552 (comment)


Whether to inject importmap for generated chunks. This importmap is used to optimize caching efficiency.
6 changes: 5 additions & 1 deletion docs/guide/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -715,7 +715,7 @@ Do not allow `data:` for [`script-src`](https://developer.mozilla.org/en-US/docs

## Build Optimizations

> Features listed below are automatically applied as part of the build process and there is no need for explicit configuration unless you want to disable them.
> Features listed below (except for the chunk importmap feature) are automatically applied as part of the build process and there is no need for explicit configuration unless you want to disable them.

### CSS Code Splitting

Expand Down Expand Up @@ -749,3 +749,7 @@ Entry ---> (A + C)
```

It is possible for `C` to have further imports, which will result in even more roundtrips in the un-optimized scenario. Vite's optimization will trace all the direct imports to completely eliminate the roundtrips regardless of import depth.

### Chunk importmap

Creating an import map for chunks helps prevent the issue of cascading cache invalidation. This import map features a list of stable file IDs linked to filenames with content-based hashes. When one chunk references another, it utilizes the file ID instead of the content-hashed filename. As a result, only the updated chunk needs cache invalidation in the browser, leaving intermediary chunks unchanged. This strategy enhances the cache hit rate following deployments.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Creating an import map for chunks helps prevent the issue of cascading cache invalidation. This import map features a list of stable file IDs linked to filenames with content-based hashes. When one chunk references another, it utilizes the file ID instead of the content-hashed filename. As a result, only the updated chunk needs cache invalidation in the browser, leaving intermediary chunks unchanged. This strategy enhances the cache hit rate following deployments.
Creating an import map for chunks helps prevent the issue of cascading cache invalidation. This import map features a list of stable file IDs linked to filenames with content-based hashes. When one chunk references another, it utilizes the file ID instead of the content-hashed filename. As a result, only the updated chunk needs cache invalidation in the browser, leaving intermediary chunks unchanged. This strategy enhances the cache hit rate following deployments.
To enable this feature, set [`build.chunkImportMap`](/config/build-options.md#build-chunkimportmap) to `true`.

84 changes: 83 additions & 1 deletion packages/plugin-legacy/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,51 @@ const _require = createRequire(import.meta.url)
const nonLeadingHashInFileNameRE = /[^/]+\[hash(?::\d+)?\]/
const prefixedHashInFileNameRE = /\W?\[hash(?::\d+)?\]/

const hashPlaceholderLeft = '!~{'
const hashPlaceholderRight = '}~'
const hashPlaceholderOverhead =
hashPlaceholderLeft.length + hashPlaceholderRight.length
const maxHashSize = 22
// from https://github.com/rollup/rollup/blob/91352494fc722bcd5e8e922cd1497b34aec57a67/src/utils/hashPlaceholders.ts#L41-L46
const hashPlaceholderRE = new RegExp(
// eslint-disable-next-line regexp/strict, regexp/prefer-w
`${hashPlaceholderLeft}[0-9a-zA-Z_$]{1,${
maxHashSize - hashPlaceholderOverhead
}}${hashPlaceholderRight}`,
'g',
)

const hashPlaceholderToFacadeModuleIdHashMap = new Map<string, string>()

function augmentFacadeModuleIdHash(name: string): string {
return name.replace(
hashPlaceholderRE,
(match) => hashPlaceholderToFacadeModuleIdHashMap.get(match) ?? match,
)
}

function createChunkImportMap(
bundle: OutputBundle,
base: string,
): Record<string, string> {
return Object.fromEntries(
Object.values(bundle)
.filter((chunk): chunk is OutputChunk => chunk.type === 'chunk')
.map((output) => {
return [
base + augmentFacadeModuleIdHash(output.preliminaryFileName),
base + output.fileName,
]
}),
)
}

export function getHash(text: Buffer | string, length = 8): string {
const h = hash('sha256', text, 'hex').substring(0, length)
if (length <= 64) return h
return h.padEnd(length, '_')
}

function viteLegacyPlugin(options: Options = {}): Plugin[] {
let config: ResolvedConfig
let targets: Options['targets']
Expand Down Expand Up @@ -167,6 +212,7 @@ function viteLegacyPlugin(options: Options = {}): Plugin[] {
const isDebug =
debugFlags.includes('vite:*') || debugFlags.includes('vite:legacy')

const chunkImportMap = new Map()
const facadeToLegacyChunkMap = new Map()
const facadeToLegacyPolyfillMap = new Map()
const facadeToModernPolyfillMap = new Map()
Expand Down Expand Up @@ -519,6 +565,16 @@ function viteLegacyPlugin(options: Options = {}): Plugin[] {
}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When build.chunkImportMap is enabled, I guess the application won't work if the browser does not support import maps but supports all features included in detectModernBrowserDetector.
I think we need to add if(HTMLScriptElement&&HTMLScriptElement.supports("importmap"))throw new Error; to detectModernBrowserDetector and modernChunkLegacyGuard.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried simply adding an if statement at the end of detectModernBrowserDetector, but it doesn't seem to have worked 👀
It seems that __vite_legacy_guard inside modernChunkLegacyGuard is not being executed from anywhere.
Could you explain how it works?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that __vite_legacy_guard inside modernChunkLegacyGuard is not being executed from anywhere.
Could you explain how it works?

The current modernChunkLegacyGuard uses the fact that browsers errors without executing a script if the script has invalid syntax (unsupported syntax). But in this case, we are judging with runtime code instead of a syntax. So I was suggesting adding the code in modernChunkLegacyGuard (i.e. changing modernChunkLegacyGuard to `if(HTMLScriptElement&&HTMLScriptElement.supports("importmap"))throw new Error;`);export function __vite_legacy_guard(){${detectModernBrowserDetector}};`.

That said, I noticed that the script that has the throw new Error won't run but still the scripts imported by that script would run. Hmm...

if (config.build.chunkImportMap) {
const hashPlaceholder = chunk.fileName.match(hashPlaceholderRE)?.[0]
if (hashPlaceholder) {
hashPlaceholderToFacadeModuleIdHashMap.set(
hashPlaceholder,
getHash(chunk.facadeModuleId ?? chunk.fileName),
)
}
}

if (!genLegacy) {
return null
}
Expand Down Expand Up @@ -574,13 +630,26 @@ function viteLegacyPlugin(options: Options = {}): Plugin[] {
return null
},

transformIndexHtml(html, { chunk }) {
transformIndexHtml(html, { chunk, filename, bundle }) {
if (config.build.ssr) return
if (!chunk) return
if (chunk.fileName.includes('-legacy')) {
// The legacy bundle is built first, and its index.html isn't actually emitted if
// modern bundle will be generated. Here we simply record its corresponding legacy chunk.
facadeToLegacyChunkMap.set(chunk.facadeModuleId, chunk.fileName)

if (config.build.chunkImportMap) {
const relativeUrlPath = path.posix.relative(
config.root,
normalizePath(filename),
)
const assetsBase = getBaseInHTML(relativeUrlPath, config)
chunkImportMap.set(
chunk.facadeModuleId,
createChunkImportMap(bundle!, assetsBase),
)
}

if (genModern) {
return
}
Expand Down Expand Up @@ -703,6 +772,19 @@ function viteLegacyPlugin(options: Options = {}): Plugin[] {
})
}

if (config.build.chunkImportMap) {
const imports = chunkImportMap.get(chunk.facadeModuleId)

if (imports) {
tags.push({
tag: 'script',
attrs: { type: 'systemjs-importmap' },
children: JSON.stringify({ imports }),
injectTo: 'head-prepend',
})
}
}

return {
html,
tags,
Expand Down
36 changes: 36 additions & 0 deletions packages/vite/src/node/__tests__/build.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,42 @@ describe('build', () => {
assertOutputHashContentChange(result[0], result[1])
})

test('file hash should not change when changes for dynamic entries while chunk map option enabled', async () => {
const buildProject = async (text: string) => {
return (await build({
root: resolve(__dirname, 'packages/build-project'),
logLevel: 'silent',
build: {
chunkImportMap: true,
write: false,
},
plugins: [
{
name: 'test',
resolveId(id) {
if (id === 'entry.js' || id === 'subentry.js') {
return '\0' + id
}
},
load(id) {
if (id === '\0entry.js') {
return `window.addEventListener('click', () => { import('subentry.js') });`
}
if (id === '\0subentry.js') {
return `console.log(${text})`
}
},
},
],
})) as RollupOutput
}

const result = await Promise.all([buildProject('foo'), buildProject('bar')])

expect(result[0].output[0].fileName).toBe(result[1].output[0].fileName)
expect(result[0].output[1].fileName).not.toBe(result[1].output[1].fileName)
})

test('file hash should change when pure css chunk changes', async () => {
const buildProject = async (cssColor: string) => {
return (await build({
Expand Down
11 changes: 11 additions & 0 deletions packages/vite/src/node/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ import { resolveEnvironmentPlugins } from './plugin'
import { manifestPlugin } from './plugins/manifest'
import type { Logger } from './logger'
import { dataURIPlugin } from './plugins/dataUri'
import { chunkImportMapPlugin } from './plugins/chunkImportMap'
import { buildImportAnalysisPlugin } from './plugins/importAnalysisBuild'
import { ssrManifestPlugin } from './ssr/ssrManifestPlugin'
import { buildLoadFallbackPlugin } from './plugins/loadFallback'
Expand Down Expand Up @@ -265,6 +266,12 @@ export interface BuildEnvironmentOptions {
* @default null
*/
watch?: WatcherOptions | null
/**
* Whether to inject importmap for generated chunks.
* This importmap is used to optimize caching efficiency.
* @default false
*/
chunkImportMap?: boolean
/**
* create the Build Environment instance
*/
Expand Down Expand Up @@ -385,6 +392,7 @@ export function resolveBuildEnvironmentOptions(
reportCompressedSize: true,
chunkSizeWarningLimit: 500,
watch: null,
chunkImportMap: false,
createEnvironment: (name, config) => new BuildEnvironment(name, config),
}

Expand Down Expand Up @@ -490,6 +498,9 @@ export async function resolveBuildPlugins(config: ResolvedConfig): Promise<{
await asyncFlatten(arraify(config.build.rollupOptions.plugins))
).filter(Boolean) as Plugin[]),
...(config.isWorker ? [webWorkerPostPlugin()] : []),
...(!config.isWorker && config.build.chunkImportMap
? [chunkImportMapPlugin()]
: []),
],
post: [
buildImportAnalysisPlugin(config),
Expand Down
117 changes: 117 additions & 0 deletions packages/vite/src/node/plugins/chunkImportMap.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
import path from 'node:path'
import type { OutputBundle, OutputChunk } from 'rollup'
import MagicString from 'magic-string'
import { getHash, normalizePath } from '../utils'
import type { Plugin } from '../plugin'
import type { ResolvedConfig } from '../config'
import type { IndexHtmlTransformHook } from './html'

const hashPlaceholderLeft = '!~{'
const hashPlaceholderRight = '}~'
const hashPlaceholderOverhead =
hashPlaceholderLeft.length + hashPlaceholderRight.length
export const maxHashSize = 22
// from https://github.com/rollup/rollup/blob/91352494fc722bcd5e8e922cd1497b34aec57a67/src/utils/hashPlaceholders.ts#L41-L46
const hashPlaceholderRE = new RegExp(
// eslint-disable-next-line regexp/strict, regexp/prefer-w
`${hashPlaceholderLeft}[0-9a-zA-Z_$]{1,${
maxHashSize - hashPlaceholderOverhead
}}${hashPlaceholderRight}`,
'g',
)

const hashPlaceholderToFacadeModuleIdHashMap = new Map()

function augmentFacadeModuleIdHash(name: string): string {
return name.replace(
hashPlaceholderRE,
(match) => hashPlaceholderToFacadeModuleIdHashMap.get(match) ?? match,
)
}

export function createChunkImportMap(
bundle: OutputBundle,
base: string = '',
): Record<string, string> {
return Object.fromEntries(
Object.values(bundle)
.filter((chunk): chunk is OutputChunk => chunk.type === 'chunk')
.map((output) => {
return [
base + augmentFacadeModuleIdHash(output.preliminaryFileName),
base + output.fileName,
]
Comment on lines +40 to +43
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Note to reviewers: the mapping by the browsers is operated after both the module specifier and the key of import maps are resolved to absolute URLs.
https://html.spec.whatwg.org/multipage/webappapis.html#import-maps:~:text=Such%20remappings%20operate,app.mjs%22

}),
)
}

export function chunkImportMapPlugin(): Plugin {
return {
name: 'vite:chunk-importmap',

// If the hash part is simply removed, there is a risk of key collisions within the importmap.
// For example, both `foo/index-[hash].js` and `index-[hash].js` would become `assets/index-.js`.
// Therefore, a hash is generated from the facadeModuleId to avoid this issue.
renderChunk(code, _chunk, _options, meta) {
Object.values(meta.chunks).forEach((chunk) => {
const hashPlaceholder = chunk.fileName.match(hashPlaceholderRE)?.[0]
if (!hashPlaceholder) return
if (hashPlaceholderToFacadeModuleIdHashMap.get(hashPlaceholder)) return

hashPlaceholderToFacadeModuleIdHashMap.set(
hashPlaceholder,
getHash(chunk.facadeModuleId ?? chunk.fileName),
)
})

const s = new MagicString(code)
// note that MagicString::replace is used instead of String::replace
s.replace(
hashPlaceholderRE,
(match) => hashPlaceholderToFacadeModuleIdHashMap.get(match) ?? match,
)
return { code: s.toString(), map: s.generateMap({ hires: 'boundary' }) }
},
}
}

export function postChunkImportMapHook(
config: ResolvedConfig,
): IndexHtmlTransformHook {
return (html, ctx) => {
if (!config.build.chunkImportMap) return

const { filename, bundle } = ctx

const relativeUrlPath = path.posix.relative(
config.root,
normalizePath(filename),
)
const assetsBase = getBaseInHTML(relativeUrlPath, config)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this does not take experimental.renderBuiltUrl into account. For non-runtime ones, we can generate a static import map. But for runtime ones, I guess we would need to output an script that constructs an import map.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven’t come up with a good way to use it together with the renderBuiltUrl runtime yet.
Considering that we need to handle the import map from other plugins, we’ll need some kind of clever solution.
Are there any other options that could be used as a reference?

I’ll need some time to investigate and think it over.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess there aren't any.


return {
html,
tags: [
{
tag: 'script',
attrs: { type: 'importmap' },
children: JSON.stringify({
imports: createChunkImportMap(bundle!, assetsBase),
}),
injectTo: 'head-prepend',
},
],
}
}
}

function getBaseInHTML(urlRelativePath: string, config: ResolvedConfig) {
// Prefer explicit URL if defined for linking to assets and public files from HTML,
// even when base relative is specified
return config.base === './' || config.base === ''
? path.posix.join(
path.posix.relative(urlRelativePath, '').slice(0, -2),
'./',
)
: config.base
}
Loading