Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: chunk importmap #16552

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/config/shared-options.md
Original file line number Diff line number Diff line change
@@ -170,6 +170,13 @@ Enabling this setting causes vite to determine file identity by the original fil

A nonce value placeholder that will be used when generating script / style tags. Setting this value will also generate a meta tag with nonce value.

## html.chunkImportMap

- **Type:** `boolean`
- **Default:** `false`

Whether to inject importmap for generated chunks. This importmap is used to optimize caching efficiency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems build.chunkImportMap is used instead of html.chunkImportMap in the code.
I think build.chunkImportMap is better as I think we should have an option to generate the import map in a separate file so that frameworks and users using backend integration can use it.
Let's move this description to docs/config/build-options.md.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might even be worth being an experimental feature. build.experimentalChunkImportMap

Im really looking forward to this change as itll be a massive win for users.

I expect though at this time there may be gotchas we need to learn and patch, as people report behaviour issues with there individual setups.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can set experimental without changing the name. 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I need to use this feature as backend integration 👍 . It would be great if we could export ImportMap contents into a file like import-maps.json, and the consumers can name as the build.manifest feature (ref: https://vite.dev/config/build-options.html#build-manifest)

## css.modules

- **Type:**
4 changes: 4 additions & 0 deletions docs/guide/features.md
Original file line number Diff line number Diff line change
@@ -712,6 +712,10 @@ By default, during build, Vite inlines small assets as data URIs. Allowing `data
Do not allow `data:` for [`script-src`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src). It will allow injection of arbitrary scripts.
:::

## Chunk importmap

Creating an importmap for chunks helps prevent the cascading cache invalidation issue. This importmap features a list of stable file id linked to filenames with content-based hashes. When one chunk references another, it utilizes the file id instead of the filename hashed by content. As a result, only the updated chunk needs cache invalidation in the browser, leaving intermediary chunks unchanged. This strategy enhances the cache hit rate following deployments.

## Build Optimizations

> Features listed below are automatically applied as part of the build process and there is no need for explicit configuration unless you want to disable them.
85 changes: 84 additions & 1 deletion packages/plugin-legacy/src/index.ts
Original file line number Diff line number Diff line change
@@ -14,6 +14,7 @@ import type {
import type {
NormalizedOutputOptions,
OutputBundle,
OutputChunk,
OutputOptions,
PreRenderedChunk,
RenderedChunk,
@@ -134,6 +135,51 @@ const _require = createRequire(import.meta.url)
const nonLeadingHashInFileNameRE = /[^/]+\[hash(?::\d+)?\]/
const prefixedHashInFileNameRE = /\W?\[hash(?::\d+)?\]/

const hashPlaceholderLeft = '!~{'
const hashPlaceholderRight = '}~'
const hashPlaceholderOverhead =
hashPlaceholderLeft.length + hashPlaceholderRight.length
const maxHashSize = 22
// from https://github.com/rollup/rollup/blob/91352494fc722bcd5e8e922cd1497b34aec57a67/src/utils/hashPlaceholders.ts#L41-L46
const hashPlaceholderRE = new RegExp(
// eslint-disable-next-line regexp/strict, regexp/prefer-w
`${hashPlaceholderLeft}[0-9a-zA-Z_$]{1,${
maxHashSize - hashPlaceholderOverhead
}}${hashPlaceholderRight}`,
'g',
)

const hashPlaceholderToFacadeModuleIdHashMap = new Map<string, string>()

function augmentFacadeModuleIdHash(name: string): string {
return name.replace(
hashPlaceholderRE,
(match) => hashPlaceholderToFacadeModuleIdHashMap.get(match) ?? match,
)
}

function createChunkImportMap(
bundle: OutputBundle,
base: string,
): Record<string, string> {
return Object.fromEntries(
Object.values(bundle)
.filter((chunk): chunk is OutputChunk => chunk.type === 'chunk')
.map((output) => {
return [
base + augmentFacadeModuleIdHash(output.preliminaryFileName),
base + output.fileName,
]
}),
)
}

export function getHash(text: Buffer | string, length = 8): string {
const h = createHash('sha256').update(text).digest('hex').substring(0, length)
if (length <= 64) return h
return h.padEnd(length, '_')
}

function viteLegacyPlugin(options: Options = {}): Plugin[] {
let config: ResolvedConfig
let targets: Options['targets']
@@ -165,6 +211,7 @@ function viteLegacyPlugin(options: Options = {}): Plugin[] {
const isDebug =
debugFlags.includes('vite:*') || debugFlags.includes('vite:legacy')

const chunkImportMap = new Map()
const facadeToLegacyChunkMap = new Map()
const facadeToLegacyPolyfillMap = new Map()
const facadeToModernPolyfillMap = new Map()
@@ -519,6 +566,16 @@ function viteLegacyPlugin(options: Options = {}): Plugin[] {
}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When build.chunkImportMap is enabled, I guess the application won't work if the browser does not support import maps but supports all features included in detectModernBrowserDetector.
I think we need to add if(HTMLScriptElement&&HTMLScriptElement.supports("importmap"))throw new Error; to detectModernBrowserDetector and modernChunkLegacyGuard.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried simply adding an if statement at the end of detectModernBrowserDetector, but it doesn't seem to have worked 👀
It seems that __vite_legacy_guard inside modernChunkLegacyGuard is not being executed from anywhere.
Could you explain how it works?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that __vite_legacy_guard inside modernChunkLegacyGuard is not being executed from anywhere.
Could you explain how it works?

The current modernChunkLegacyGuard uses the fact that browsers errors without executing a script if the script has invalid syntax (unsupported syntax). But in this case, we are judging with runtime code instead of a syntax. So I was suggesting adding the code in modernChunkLegacyGuard (i.e. changing modernChunkLegacyGuard to `if(HTMLScriptElement&&HTMLScriptElement.supports("importmap"))throw new Error;`);export function __vite_legacy_guard(){${detectModernBrowserDetector}};`.

That said, I noticed that the script that has the throw new Error won't run but still the scripts imported by that script would run. Hmm...

if (config.build.chunkImportMap) {
const hashPlaceholder = chunk.fileName.match(hashPlaceholderRE)?.[0]
if (hashPlaceholder) {
hashPlaceholderToFacadeModuleIdHashMap.set(
hashPlaceholder,
getHash(chunk.facadeModuleId ?? chunk.fileName),
)
}
}

if (!genLegacy) {
return null
}
@@ -574,13 +631,26 @@ function viteLegacyPlugin(options: Options = {}): Plugin[] {
return null
},

transformIndexHtml(html, { chunk }) {
transformIndexHtml(html, { chunk, filename, bundle }) {
if (config.build.ssr) return
if (!chunk) return
if (chunk.fileName.includes('-legacy')) {
// The legacy bundle is built first, and its index.html isn't actually emitted if
// modern bundle will be generated. Here we simply record its corresponding legacy chunk.
facadeToLegacyChunkMap.set(chunk.facadeModuleId, chunk.fileName)

if (config.build.chunkImportMap) {
const relativeUrlPath = path.posix.relative(
config.root,
normalizePath(filename),
)
const assetsBase = getBaseInHTML(relativeUrlPath, config)
chunkImportMap.set(
chunk.facadeModuleId,
createChunkImportMap(bundle!, assetsBase),
)
}

if (genModern) {
return
}
@@ -703,6 +773,19 @@ function viteLegacyPlugin(options: Options = {}): Plugin[] {
})
}

if (config.build.chunkImportMap) {
const imports = chunkImportMap.get(chunk.facadeModuleId)

if (imports) {
tags.push({
tag: 'script',
attrs: { type: 'systemjs-importmap' },
children: JSON.stringify({ imports }),
injectTo: 'head-prepend',
})
}
}

return {
html,
tags,
36 changes: 36 additions & 0 deletions packages/vite/src/node/__tests__/build.spec.ts
Original file line number Diff line number Diff line change
@@ -60,6 +60,42 @@ describe('build', () => {
assertOutputHashContentChange(result[0], result[1])
})

test('file hash should not change when changes for dynamic entries while chunk map option enabled', async () => {
const buildProject = async (text: string) => {
return (await build({
root: resolve(__dirname, 'packages/build-project'),
logLevel: 'silent',
build: {
chunkImportMap: true,
write: false,
},
plugins: [
{
name: 'test',
resolveId(id) {
if (id === 'entry.js' || id === 'subentry.js') {
return '\0' + id
}
},
load(id) {
if (id === '\0entry.js') {
return `window.addEventListener('click', () => { import('subentry.js') });`
}
if (id === '\0subentry.js') {
return `console.log(${text})`
}
},
},
],
})) as RollupOutput
}

const result = await Promise.all([buildProject('foo'), buildProject('bar')])

expect(result[0].output[0].fileName).toBe(result[1].output[0].fileName)
expect(result[0].output[1].fileName).not.toBe(result[1].output[1].fileName)
})

test('file hash should change when pure css chunk changes', async () => {
const buildProject = async (cssColor: string) => {
return (await build({
11 changes: 11 additions & 0 deletions packages/vite/src/node/build.ts
Original file line number Diff line number Diff line change
@@ -53,6 +53,7 @@ import { resolveEnvironmentPlugins } from './plugin'
import { manifestPlugin } from './plugins/manifest'
import type { Logger } from './logger'
import { dataURIPlugin } from './plugins/dataUri'
import { chunkImportMapPlugin } from './plugins/chunkImportMap'
import { buildImportAnalysisPlugin } from './plugins/importAnalysisBuild'
import { ssrManifestPlugin } from './ssr/ssrManifestPlugin'
import { buildLoadFallbackPlugin } from './plugins/loadFallback'
@@ -262,6 +263,12 @@ export interface BuildEnvironmentOptions {
* @default null
*/
watch?: WatcherOptions | null
/**
* Whether to inject importmap for generated chunks.
* This importmap is used to optimize caching efficiency.
* @default false
*/
chunkImportMap?: boolean
/**
* create the Build Environment instance
*/
@@ -382,6 +389,7 @@ export function resolveBuildEnvironmentOptions(
reportCompressedSize: true,
chunkSizeWarningLimit: 500,
watch: null,
chunkImportMap: false,
createEnvironment: (name, config) => new BuildEnvironment(name, config),
}

@@ -487,6 +495,9 @@ export async function resolveBuildPlugins(config: ResolvedConfig): Promise<{
await asyncFlatten(arraify(config.build.rollupOptions.plugins))
).filter(Boolean) as Plugin[]),
...(config.isWorker ? [webWorkerPostPlugin()] : []),
...(!config.isWorker && config.build.chunkImportMap
? [chunkImportMapPlugin()]
: []),
],
post: [
buildImportAnalysisPlugin(config),
117 changes: 117 additions & 0 deletions packages/vite/src/node/plugins/chunkImportMap.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
import path from 'node:path'
import type { OutputBundle, OutputChunk } from 'rollup'
import MagicString from 'magic-string'
import { getHash, normalizePath } from '../utils'
import type { Plugin } from '../plugin'
import type { ResolvedConfig } from '../config'
import type { IndexHtmlTransformHook } from './html'

const hashPlaceholderLeft = '!~{'
const hashPlaceholderRight = '}~'
const hashPlaceholderOverhead =
hashPlaceholderLeft.length + hashPlaceholderRight.length
export const maxHashSize = 22
// from https://github.com/rollup/rollup/blob/91352494fc722bcd5e8e922cd1497b34aec57a67/src/utils/hashPlaceholders.ts#L41-L46
const hashPlaceholderRE = new RegExp(
// eslint-disable-next-line regexp/strict, regexp/prefer-w
`${hashPlaceholderLeft}[0-9a-zA-Z_$]{1,${
maxHashSize - hashPlaceholderOverhead
}}${hashPlaceholderRight}`,
'g',
)

const hashPlaceholderToFacadeModuleIdHashMap = new Map()

function augmentFacadeModuleIdHash(name: string): string {
return name.replace(
hashPlaceholderRE,
(match) => hashPlaceholderToFacadeModuleIdHashMap.get(match) ?? match,
)
}

export function createChunkImportMap(
bundle: OutputBundle,
base: string = '',
): Record<string, string> {
return Object.fromEntries(
Object.values(bundle)
.filter((chunk): chunk is OutputChunk => chunk.type === 'chunk')
.map((output) => {
return [
base + augmentFacadeModuleIdHash(output.preliminaryFileName),
base + output.fileName,
]
Comment on lines +40 to +43
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Note to reviewers: the mapping by the browsers is operated after both the module specifier and the key of import maps are resolved to absolute URLs.
https://html.spec.whatwg.org/multipage/webappapis.html#import-maps:~:text=Such%20remappings%20operate,app.mjs%22

}),
)
}

export function chunkImportMapPlugin(): Plugin {
return {
name: 'vite:chunk-importmap',

// If the hash part is simply removed, there is a risk of key collisions within the importmap.
// For example, both `foo/index-[hash].js` and `index-[hash].js` would become `assets/index-.js`.
// Therefore, a hash is generated from the facadeModuleId to avoid this issue.
renderChunk(code, _chunk, _options, meta) {
Object.values(meta.chunks).forEach((chunk) => {
const hashPlaceholder = chunk.fileName.match(hashPlaceholderRE)?.[0]
if (!hashPlaceholder) return
if (hashPlaceholderToFacadeModuleIdHashMap.get(hashPlaceholder)) return

hashPlaceholderToFacadeModuleIdHashMap.set(
hashPlaceholder,
getHash(chunk.facadeModuleId ?? chunk.fileName),
)
})

const codeProcessed = augmentFacadeModuleIdHash(code)
return {
code: codeProcessed,
map: new MagicString(codeProcessed).generateMap({
hires: 'boundary',
}),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 For reviewers: This works as rollup calculates the chunk hash after the renderChunk hooks are run since Rollup 3.

Reference: https://www.youtube.com/watch?v=cFwO9UvDzfI

}
},
}
}

export function postChunkImportMapHook(
config: ResolvedConfig,
): IndexHtmlTransformHook {
return (html, ctx) => {
if (!config.build.chunkImportMap) return

const { filename, bundle } = ctx

const relativeUrlPath = path.posix.relative(
config.root,
normalizePath(filename),
)
const assetsBase = getBaseInHTML(relativeUrlPath, config)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this does not take experimental.renderBuiltUrl into account. For non-runtime ones, we can generate a static import map. But for runtime ones, I guess we would need to output an script that constructs an import map.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven’t come up with a good way to use it together with the renderBuiltUrl runtime yet.
Considering that we need to handle the import map from other plugins, we’ll need some kind of clever solution.
Are there any other options that could be used as a reference?

I’ll need some time to investigate and think it over.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess there aren't any.


return {
html,
tags: [
{
tag: 'script',
attrs: { type: 'importmap' },
children: JSON.stringify({
imports: createChunkImportMap(bundle!, assetsBase),
}),
injectTo: 'head-prepend',
},
],
}
}
}

function getBaseInHTML(urlRelativePath: string, config: ResolvedConfig) {
// Prefer explicit URL if defined for linking to assets and public files from HTML,
// even when base relative is specified
return config.base === './' || config.base === ''
? path.posix.join(
path.posix.relative(urlRelativePath, '').slice(0, -2),
'./',
)
: config.base
}
Loading