Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(deps)!: migrate fast-glob to tinyglobby #18243

Merged
merged 21 commits into from
Oct 17, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/config/dep-optimization-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

By default, Vite will crawl all your `.html` files to detect dependencies that need to be pre-bundled (ignoring `node_modules`, `build.outDir`, `__tests__` and `coverage`). If `build.rollupOptions.input` is specified, Vite will crawl those entry points instead.

If neither of these fit your needs, you can specify custom entries using this option - the value should be a [fast-glob pattern](https://github.com/mrmlnc/fast-glob#basic-syntax) or array of patterns that are relative from Vite project root. This will overwrite default entries inference. Only `node_modules` and `build.outDir` folders will be ignored by default when `optimizeDeps.entries` is explicitly defined. If other folders need to be ignored, you can use an ignore pattern as part of the entries list, marked with an initial `!`. If you don't want to ignore `node_modules` and `build.outDir`, you can specify using literal string paths (without fast-glob patterns) instead.
If neither of these fit your needs, you can specify custom entries using this option - the value should be a [`tinyglobby` pattern](https://github.com/SuperchupuDev/tinyglobby) or array of patterns that are relative from Vite project root. This will overwrite default entries inference. Only `node_modules` and `build.outDir` folders will be ignored by default when `optimizeDeps.entries` is explicitly defined. If other folders need to be ignored, you can use an ignore pattern as part of the entries list, marked with an initial `!`. If you don't want to ignore `node_modules` and `build.outDir`, you can specify using literal string paths (without `tinyglobby` patterns) instead.

## optimizeDeps.exclude

Expand Down
2 changes: 1 addition & 1 deletion docs/config/server-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ The error that appears in the Browser when the fallback happens can be ignored.

Warm up files to transform and cache the results in advance. This improves the initial page load during server starts and prevents transform waterfalls.

`clientFiles` are files that are used in the client only, while `ssrFiles` are files that are used in SSR only. They accept an array of file paths or [`fast-glob`](https://github.com/mrmlnc/fast-glob) patterns relative to the `root`.
`clientFiles` are files that are used in the client only, while `ssrFiles` are files that are used in SSR only. They accept an array of file paths or [`tinyglobby`](https://github.com/SuperchupuDev/tinyglobby) patterns relative to the `root`.

Make sure to only add files that are frequently used to not overload the Vite dev server on startup.

Expand Down
2 changes: 1 addition & 1 deletion docs/guide/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -542,7 +542,7 @@ Note that:

- This is a Vite-only feature and is not a web or ES standard.
- The glob patterns are treated like import specifiers: they must be either relative (start with `./`) or absolute (start with `/`, resolved relative to project root) or an alias path (see [`resolve.alias` option](/config/shared-options.md#resolve-alias)).
- The glob matching is done via [`fast-glob`](https://github.com/mrmlnc/fast-glob) - check out its documentation for [supported glob patterns](https://github.com/mrmlnc/fast-glob#pattern-syntax).
- The glob matching is done via [`tinyglobby`](https://github.com/SuperchupuDev/tinyglobby).
- You should also be aware that all the arguments in the `import.meta.glob` must be **passed as literals**. You can NOT use variables or expressions in them.

## Dynamic Import
Expand Down
2 changes: 1 addition & 1 deletion packages/vite/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,6 @@
"escape-html": "^1.0.3",
"estree-walker": "^3.0.3",
"etag": "^1.8.1",
"fast-glob": "^3.3.2",
"http-proxy": "^1.18.1",
"launch-editor-middleware": "^2.9.1",
"lightningcss": "^1.27.0",
Expand Down Expand Up @@ -148,6 +147,7 @@
"source-map-support": "^0.5.21",
"strip-ansi": "^7.1.0",
"strip-literal": "^2.1.0",
"tinyglobby": "^0.2.7",
"tsconfck": "^3.1.3",
"tslib": "^2.7.0",
"types": "link:./types",
Expand Down
8 changes: 4 additions & 4 deletions packages/vite/src/node/optimizer/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import colors from 'picocolors'
import type { BuildContext, BuildOptions as EsbuildBuildOptions } from 'esbuild'
import esbuild, { build } from 'esbuild'
import { init, parse } from 'es-module-lexer'
import glob from 'fast-glob'
import { isDynamicPattern } from 'tinyglobby'
import type { ResolvedConfig } from '../config'
import {
createDebugger,
Expand Down Expand Up @@ -151,8 +151,8 @@ export type DepOptimizationOptions = DepOptimizationConfig & {
* will crawl those entry points instead.
*
* If neither of these fit your needs, you can specify custom entries using
* this option - the value should be a fast-glob pattern or array of patterns
* (https://github.com/mrmlnc/fast-glob#basic-syntax) that are relative from
* this option - the value should be a tinyglobby pattern or array of patterns
* (https://github.com/SuperchupuDev/tinyglobby) that are relative from
* vite project root. This will overwrite default entries inference.
*/
entries?: string | string[]
Expand Down Expand Up @@ -826,7 +826,7 @@ export async function addManuallyIncludedOptimizeDeps(
const includes = [...optimizeDepsInclude]
for (let i = 0; i < includes.length; i++) {
const id = includes[i]
if (glob.isDynamicPattern(id)) {
if (isDynamicPattern(id)) {
const globIds = expandGlobIds(id, environment.getTopLevelConfig())
includes.splice(i, 1, ...globIds)
i += globIds.length - 1
Expand Down
31 changes: 12 additions & 19 deletions packages/vite/src/node/optimizer/resolve.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import path from 'node:path'
import glob from 'fast-glob'
import micromatch from 'micromatch'
import { globSync } from 'tinyglobby'
import type { ResolvedConfig } from '../config'
import { escapeRegex, getNpmPackageName } from '../utils'
import { resolvePackageData } from '../packages'
Expand Down Expand Up @@ -79,6 +79,7 @@ export function expandGlobIds(id: string, config: ResolvedConfig): string[] {
const exportsValue = getFirstExportStringValue(exports[key])
if (!exportsValue) continue

// TODO: Raise during the PR review.
// "./dist/glob/*-browser/*.js" => "./dist/glob/**/*-browser/**/*.js"
// NOTE: in some cases, this could expand to consecutive /**/*/**/* etc
// but it's fine since fast-glob handles it the same.
Expand All @@ -89,22 +90,12 @@ export function expandGlobIds(id: string, config: ResolvedConfig): string[] {
)

possibleExportPaths.push(
...glob
.sync(exportValuePattern, {
cwd: pkgData.dir,
ignore: ['node_modules'],
})
...globSync(exportValuePattern, {
cwd: pkgData.dir,
expandDirectories: false,
ignore: ['node_modules'],
})
.map((filePath) => {
// ensure "./" prefix for inconsistent fast-glob result
// glob.sync("./some-dir/**/*") -> "./some-dir/some-file"
// glob.sync("./**/*") -> "some-dir/some-file"
if (
exportsValue.startsWith('./') &&
!filePath.startsWith('./')
) {
filePath = './' + filePath
}

// "./glob/*": "./dist/glob/*-browser/*.js"
// `filePath`: "./dist/glob/foo-browser/foo.js"
// we need to revert the file path back to the export key by
Expand Down Expand Up @@ -146,9 +137,11 @@ export function expandGlobIds(id: string, config: ResolvedConfig): string[] {
return matched
} else {
// for packages without exports, we can do a simple glob
const matched = glob
.sync(pattern, { cwd: pkgData.dir, ignore: ['node_modules'] })
.map((match) => path.posix.join(pkgName, slash(match)))
const matched = globSync(pattern, {
cwd: pkgData.dir,
expandDirectories: false,
ignore: ['node_modules'],
}).map((match) => path.posix.join(pkgName, slash(match)))
matched.unshift(pkgName)
return matched
}
Expand Down
7 changes: 3 additions & 4 deletions packages/vite/src/node/optimizer/scan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ import fs from 'node:fs'
import fsp from 'node:fs/promises'
import path from 'node:path'
import { performance } from 'node:perf_hooks'
import glob from 'fast-glob'
import type {
BuildContext,
Loader,
Expand All @@ -13,6 +12,7 @@ import type {
import esbuild, { formatMessages, transform } from 'esbuild'
import type { PartialResolvedId } from 'rollup'
import colors from 'picocolors'
import { glob, isDynamicPattern } from 'tinyglobby'
import {
CSS_LANGS_RE,
JS_TYPES_RE,
Expand Down Expand Up @@ -332,12 +332,13 @@ function orderedDependencies(deps: Record<string, string>) {

function globEntries(pattern: string | string[], environment: ScanEnvironment) {
const resolvedPatterns = arraify(pattern)
if (resolvedPatterns.every((str) => !glob.isDynamicPattern(str))) {
if (resolvedPatterns.every((str) => !isDynamicPattern(str))) {
return resolvedPatterns.map((p) =>
normalizePath(path.resolve(environment.config.root, p)),
)
}
return glob(pattern, {
absolute: true,
cwd: environment.config.root,
ignore: [
'**/node_modules/**',
Expand All @@ -347,8 +348,6 @@ function globEntries(pattern: string | string[], environment: ScanEnvironment) {
? []
: [`**/__tests__/**`, `**/coverage/**`]),
],
absolute: true,
suppressErrors: true, // suppress EACCES errors
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such errors are suppressed by default in tinyglobby.

})
}

Expand Down
7 changes: 4 additions & 3 deletions packages/vite/src/node/plugins/css.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ import fsp from 'node:fs/promises'
import path from 'node:path'
import { createRequire } from 'node:module'
import { fileURLToPath, pathToFileURL } from 'node:url'
import glob from 'fast-glob'
import postcssrc from 'postcss-load-config'
import type {
ExistingRawSourceMap,
Expand All @@ -27,6 +26,7 @@ import type { TransformOptions } from 'esbuild'
import { formatMessages, transform } from 'esbuild'
import type { RawSourceMap } from '@ampproject/remapping'
import { WorkerWithFallback } from 'artichokie'
import { escapePath, globSync } from 'tinyglobby'
import { getCodeWithSourcemap, injectSourcesContent } from '../server/sourcemap'
import type { EnvironmentModuleNode } from '../server/moduleGraph'
import {
Expand Down Expand Up @@ -1406,10 +1406,11 @@ async function compileCSS(
// https://github.com/postcss/postcss/blob/main/docs/guidelines/plugin.md#3-dependencies
const { dir, glob: globPattern = '**' } = message
const pattern =
glob.escapePath(normalizePath(path.resolve(path.dirname(id), dir))) +
escapePath(normalizePath(path.resolve(path.dirname(id), dir))) +
`/` +
globPattern
const files = glob.sync(pattern, {
const files = globSync(pattern, {
expandDirectories: false,
ignore: ['**/node_modules/**'],
})
for (let i = 0; i < files.length; i++) {
Expand Down
10 changes: 5 additions & 5 deletions packages/vite/src/node/plugins/importMetaGlob.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ import type {
} from 'estree'
import type { CustomPluginOptions, RollupAstNode, RollupError } from 'rollup'
import MagicString from 'magic-string'
import fg from 'fast-glob'
import { stringifyQuery } from 'ufo'
import type { GeneralImportGlobOptions } from 'types/importGlob'
import { parseAstAsync } from 'rollup/parseAst'
import { escapePath, glob } from 'tinyglobby'
import type { Plugin } from '../plugin'
import type { EnvironmentModuleNode } from '../server/moduleGraph'
import type { ResolvedConfig } from '../config'
Expand Down Expand Up @@ -396,10 +396,11 @@ export async function transformGlobImport(
async ({ globsResolved, isRelative, options, index, start, end }) => {
const cwd = getCommonBase(globsResolved) ?? root
const files = (
await fg(globsResolved, {
cwd,
await glob(globsResolved, {
absolute: true,
cwd,
dot: !!options.exhaustive,
expandDirectories: false,
ignore: options.exhaustive
? []
: [join(cwd, '**/node_modules/**')],
Expand Down Expand Up @@ -515,8 +516,7 @@ type IdResolver = (

function globSafePath(path: string) {
// slash path to ensure \ is converted to / as \ could lead to a double escape scenario
// see https://github.com/mrmlnc/fast-glob#advanced-syntax
return fg.escapePath(normalizePath(path))
return escapePath(normalizePath(path))
}

function lastNthChar(str: string, n: number) {
Expand Down
5 changes: 3 additions & 2 deletions packages/vite/src/node/server/warmup.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import fs from 'node:fs/promises'
import path from 'node:path'
import glob from 'fast-glob'
import colors from 'picocolors'
import { glob } from 'tinyglobby'
import { FS_PREFIX } from '../constants'
import { normalizePath } from '../utils'
import type { ViteDevServer } from '../index'
Expand Down Expand Up @@ -71,7 +71,8 @@ function fileToUrl(file: string, root: string) {

function mapFiles(files: string[], root: string) {
return glob(files, {
cwd: root,
absolute: true,
cwd: root,
expandDirectories: false,
})
}
6 changes: 3 additions & 3 deletions packages/vite/src/node/watch.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { EventEmitter } from 'node:events'
import path from 'node:path'
import glob from 'fast-glob'
import type { FSWatcher, WatchOptions } from 'dep-types/chokidar'
import type { OutputOptions } from 'rollup'
import colors from 'picocolors'
import { escapePath } from 'tinyglobby'
import { withTrailingSlash } from '../shared/utils'
import { arraify, normalizePath } from './utils'
import type { Logger } from './logger'
Expand Down Expand Up @@ -59,12 +59,12 @@ export function resolveChokidarOptions(
'**/.git/**',
'**/node_modules/**',
'**/test-results/**', // Playwright
glob.escapePath(cacheDir) + '/**',
escapePath(cacheDir) + '/**',
...arraify(ignoredList || []),
]
if (emptyOutDir) {
ignored.push(
...[...resolvedOutDirs].map((outDir) => glob.escapePath(outDir) + '/**'),
...[...resolvedOutDirs].map((outDir) => escapePath(outDir) + '/**'),
)
}

Expand Down
2 changes: 1 addition & 1 deletion playground/backend-integration/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,6 @@
"devDependencies": {
"sass": "^1.79.4",
"tailwindcss": "^3.4.13",
"fast-glob": "^3.3.2"
"tinyglobby": "^0.2.7"
}
}
9 changes: 5 additions & 4 deletions playground/backend-integration/vite.config.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import path from 'node:path'
import glob from 'fast-glob'
import { globSync } from 'tinyglobby'
import { defineConfig, normalizePath } from 'vite'

/**
Expand All @@ -14,9 +14,10 @@ function BackendIntegrationExample() {
const root = path.join(sourceCodeDir, 'entrypoints')
const outDir = path.relative(root, path.join(projectRoot, 'dist/dev'))

const entrypoints = glob
.sync(`${normalizePath(root)}/**/*`, { onlyFiles: true })
.map((filename) => [path.relative(root, filename), filename])
const entrypoints = globSync(`${normalizePath(root)}/**/*`, {
expandDirectories: false,
onlyFiles: true,
}).map((filename) => [path.relative(root, filename), filename])

entrypoints.push(['tailwindcss-colors', 'tailwindcss/colors.js'])
entrypoints.push(['bar.css', path.resolve(__dirname, './dir/foo.css')])
Expand Down
4 changes: 2 additions & 2 deletions playground/css/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
"@vitejs/test-css-js-dep": "file:./css-js-dep",
"@vitejs/test-css-proxy-dep": "file:./css-proxy-dep",
"@vitejs/test-scss-proxy-dep": "file:./scss-proxy-dep",
"fast-glob": "^3.3.2",
"less": "^4.2.0",
"postcss-nested": "^6.2.0",
"sass": "^1.79.4",
"stylus": "^0.63.0",
"sugarss": "^4.0.1"
"sugarss": "^4.0.1",
"tinyglobby": "^0.2.7"
},
"imports": {
"#imports": "./imports-field.css"
Expand Down
4 changes: 2 additions & 2 deletions playground/css/postcss.config.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import fs from 'node:fs'
import path from 'node:path'
import glob from 'fast-glob'
import { globSync } from 'tinyglobby'
import { normalizePath } from 'vite'
import postcssNested from 'postcss-nested'

Expand All @@ -19,7 +19,7 @@ function testDirDep() {
const pattern = normalizePath(
path.resolve(path.dirname(result.opts.from), './glob-dep/**/*.css'),
)
const files = glob.sync(pattern)
const files = globSync(pattern, { expandDirectories: false })
const text = files.map((f) => fs.readFileSync(f, 'utf-8')).join('\n')
atRule.parent.insertAfter(atRule, text)
atRule.remove()
Expand Down
Loading
Loading