Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions agent/client/src/Agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,9 @@ export class Agent {
}> {
this.karton = await createKartonServer<KartonContract>({
procedures: {
fuzzyFileSearch: async (searchString) => {
return this.clientRuntime.fileSystem.fuzzySearch(searchString);
},
undoToolCallsUntilUserMessage: async (userMessageId, chatId) => {
await this.undoToolCallsUntilUserMessage(userMessageId, chatId);
},
Expand Down
3 changes: 2 additions & 1 deletion agent/runtime-interface/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
"devDependencies": {
"rimraf": "^5.0.0",
"typescript": "^5.0.0",
"zod": "4.0.17"
"zod": "4.0.17",
"fuse.js": "^7.1.0"
}
}
21 changes: 21 additions & 0 deletions agent/runtime-interface/src/file-operations.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,17 @@
import { z } from 'zod';
import type { FuseResult } from 'fuse.js';

export type FileData = {
filepath: string;
filename: string;
dirname: string;
fullpath: string;
};

export const FuzzyFileSearchInputSchema = z.object({
searchString: z.string(),
});
export type FuzzyFileSearchResult = FuseResult<FileData>[];

/**
* Core file system operation results
Expand Down Expand Up @@ -123,6 +136,13 @@ export interface IFileSystemProvider {
*/
writeFile(path: string, content: string): Promise<FileOperationResult>;

/**
* Fuzzy searches for files in the file system.
* @param searchString - The string to search for
* @returns Fuzzy search results
*/
fuzzySearch(searchString: string): Promise<FuzzyFileSearchResult>;

/**
* Edits a file by replacing content between specified lines.
* More versatile than insertLines/overwriteLines.
Expand Down Expand Up @@ -416,6 +436,7 @@ export abstract class BaseFileSystemProvider implements IFileSystemProvider {
dryRun?: boolean;
},
): Promise<SearchReplaceResult>;
abstract fuzzySearch(searchString: string): Promise<FuzzyFileSearchResult>;
abstract fileExists(path: string): Promise<boolean>;
abstract isDirectory(path: string): Promise<boolean>;
abstract getFileStats(
Expand Down
2 changes: 2 additions & 0 deletions agent/runtime-interface/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ export {
type GlobResult,
type SearchReplaceMatch,
type SearchReplaceResult,
type FuzzyFileSearchResult,
type FileData,
} from './file-operations.js';

export interface ClientRuntime {
Expand Down
1 change: 1 addition & 0 deletions agent/runtime-mock/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
"type-check": "tsc --noEmit"
},
"dependencies": {
"fuse.js": "^7.1.0",
"memfs": "^4.17.2"
},
"devDependencies": {
Expand Down
144 changes: 144 additions & 0 deletions agent/runtime-mock/src/mock-file-system.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { Volume } from 'memfs';
import type { DirectoryJSON } from 'memfs';
import { basename, dirname } from 'node:path';
import Fuse, { type IFuseOptions } from 'fuse.js';
import {
Comment on lines +3 to 5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Use POSIX path helpers with memfs to avoid Windows separator issues

Memfs is POSIX-like; node:path may emit backslashes on Windows. Use node:path/posix for consistency across platforms.

-import { basename, dirname } from 'node:path';
+import { basename, dirname } from 'node:path/posix';
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import { basename, dirname } from 'node:path';
import Fuse, { type IFuseOptions } from 'fuse.js';
import {
import { basename, dirname } from 'node:path/posix';
import Fuse, { type IFuseOptions } from 'fuse.js';
import {
🤖 Prompt for AI Agents
In agent/runtime-mock/src/mock-file-system.ts around lines 3 to 5, the file
imports basename and dirname from 'node:path', which can produce backslashes on
Windows; replace that import with the POSIX path helpers by importing basename
and dirname from 'node:path/posix' (or import * as posix from 'node:path/posix'
and use posix.basename/posix.dirname) so memfs receives consistent POSIX-style
paths across platforms.

BaseFileSystemProvider,
type FileSystemProviderConfig,
Expand All @@ -12,6 +14,8 @@ import {
type GlobResult,
type SearchReplaceResult,
type SearchReplaceMatch,
type FileData,
type FuzzyFileSearchResult,
} from '@stagewise/agent-runtime-interface';

export interface MockFileSystemConfig extends FileSystemProviderConfig {
Expand All @@ -27,6 +31,12 @@ export interface MockFileSystemConfig extends FileSystemProviderConfig {
gitignorePatterns?: string[];
}

type FileSearchCache = {
files: FileData[];
fuse: Fuse<FileData>;
lastUpdated: number;
};

/**
* Mock file system provider using memfs for testing
* Implements all IFileSystemProvider methods for comprehensive testing
Expand All @@ -35,6 +45,11 @@ export class MockFileSystemProvider extends BaseFileSystemProvider {
private volume: Volume;
private gitignorePatterns: string[] = [];

// Fuzzy search cache
private fileSearchCache: FileSearchCache | null = null;
private readonly CACHE_TTL = 30000; // 30 seconds cache TTL
private fileIndexPromise: Promise<void> | null = null;

constructor(config: MockFileSystemConfig = { workingDirectory: '/test' }) {
super(config);
this.gitignorePatterns = config.gitignorePatterns || [];
Expand Down Expand Up @@ -466,6 +481,135 @@ export class MockFileSystemProvider extends BaseFileSystemProvider {
}
}

async fuzzySearch(searchString: string): Promise<FuzzyFileSearchResult> {
// Build or refresh the file index if needed
await this.ensureFileIndexBuilt();

if (!this.fileSearchCache) {
// This shouldn't happen after ensureFileIndexBuilt, but handle it gracefully
return [];
}

// Perform the fuzzy search
const results = this.fileSearchCache.fuse.search(searchString);
return results;
}

private async ensureFileIndexBuilt(): Promise<void> {
const now = Date.now();

// Check if cache is valid
if (
this.fileSearchCache &&
now - this.fileSearchCache.lastUpdated < this.CACHE_TTL
) {
return;
}

// If already building, wait for it
if (this.fileIndexPromise) {
await this.fileIndexPromise;
return;
}

// Build the index
this.fileIndexPromise = this.buildFileIndex();
try {
await this.fileIndexPromise;
} finally {
this.fileIndexPromise = null;
}
}

private async buildFileIndex(): Promise<void> {
try {
const fileDataList: FileData[] = [];

// Recursively traverse the volume to get all files
const traverseVolume = (currentPath: string): void => {
try {
const items = this.volume.readdirSync(currentPath) as string[];

for (const item of items) {
const itemPath = this.joinPaths(currentPath, item);
const relativePath = this.getRelativePath(
this.config.workingDirectory,
itemPath,
);

// Skip gitignored files
if (this.shouldIgnorePattern(relativePath)) {
continue;
}

try {
const stats = this.volume.statSync(itemPath);

if (stats.isFile()) {
// Add file to the list
fileDataList.push({
filepath: relativePath,
filename: basename(itemPath),
dirname: dirname(relativePath),
fullpath: itemPath,
});
} else if (stats.isDirectory()) {
// Recurse into subdirectory
traverseVolume(itemPath);
}
} catch {
// Skip items that can't be stat'd
}
}
} catch {
// Skip directories that can't be read
}
};

// Start traversal from working directory
traverseVolume(this.config.workingDirectory);

// Configure Fuse for optimal file search
const fuseOptions: IFuseOptions<FileData> = {
keys: [
{ name: 'filename', weight: 2.0 }, // Filename has highest priority
{ name: 'filepath', weight: 1.0 }, // Full path has secondary priority
{ name: 'dirname', weight: 0.5 }, // Directory has lower priority
],
threshold: 0.4, // Adjust for sensitivity (0 = exact, 1 = match anything)
includeScore: true,
includeMatches: true,
minMatchCharLength: 1,
shouldSort: true,
findAllMatches: false,
location: 0,
distance: 100,
useExtendedSearch: false,
ignoreLocation: false,
ignoreFieldNorm: false,
fieldNormWeight: 1,
};

// Create the Fuse instance
const fuse = new Fuse(fileDataList, fuseOptions);

// Update the cache
this.fileSearchCache = {
files: fileDataList,
fuse: fuse,
lastUpdated: Date.now(),
};
} catch (error) {
// Log error and create an empty cache to prevent repeated failures
console.error('[fuzzySearch] Error building file index:', error);
this.fileSearchCache = {
files: [],
fuse: new Fuse([], {}),
lastUpdated: Date.now(),
};
}
}

async searchAndReplace(
filePath: string,
searchString: string,
Expand Down
29 changes: 28 additions & 1 deletion agent/runtime-node/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,34 @@ const buildOptions: BuildOptions = {
format: 'esm',
platform: 'node',
target: 'node16',
external: ['vscode'], // VSCode extension API should always be external
external: [
'vscode', // VSCode extension API should always be external
// Node.js built-in modules
'os',
'path',
'fs',
'util',
'stream',
'events',
'buffer',
'crypto',
'child_process',
'http',
'https',
'url',
'querystring',
'assert',
'zlib',
'tty',
'net',
'dns',
// Dependencies to prevent bundling issues
'fast-glob',
'fuse.js',
'ignore',
'minimatch',
'@stagewise/agent-runtime-interface',
],
Comment on lines +13 to +40
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

External list: compute Node built-ins programmatically and bundle app deps (avoid over-externalizing).

  • Hardcoding built-ins is redundant and misses node:-prefixed specifiers (e.g., node:path) used by the new code; esbuild treats Node built-ins as external by default, but if you want explicitness, generate both forms from builtinModules.
  • Per our learning, keep workspace packages external, but bundle most third-party deps; externalizing fast-glob, fuse.js, ignore, minimatch shifts breakage to runtime and can cause “module not found” unless pinned in deps. The summary only mentions fast-glob/fuse.js added—please verify ignore/minimatch are declared if they’re truly needed externally.

Apply the following changes:

  1. Add this near the imports (outside the selected range):
import { builtinModules } from 'node:module';

const nodeBuiltins = Array.from(
  new Set([...builtinModules, ...builtinModules.map((m) => `node:${m}`)])
);
  1. Replace the externals block to rely on the generated list and stop externalizing app deps:
-  external: [
-    'vscode', // VSCode extension API should always be external
-    // Node.js built-in modules
-    'os',
-    'path',
-    'fs',
-    'util',
-    'stream',
-    'events',
-    'buffer',
-    'crypto',
-    'child_process',
-    'http',
-    'https',
-    'url',
-    'querystring',
-    'assert',
-    'zlib',
-    'tty',
-    'net',
-    'dns',
-    // Dependencies to prevent bundling issues
-    'fast-glob',
-    'fuse.js',
-    'ignore',
-    'minimatch',
-    '@stagewise/agent-runtime-interface',
-  ],
+  external: [
+    'vscode',
+    ...nodeBuiltins,
+    '@stagewise/agent-runtime-interface',
+  ],

If you must keep any third-party lib external due to a demonstrated bundling issue, document the reason in a code comment and ensure it’s listed in dependencies.

Run to verify imports and deps:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Do we use node: built-ins?
rg -nP --type=ts 'from\s+["'\'']node:' | sed 's/^/node-prefix import: /'

# 2) Are externals declared as runtime-node deps?
RNPKG="$(fd -p package.json -t f agent/runtime-node | head -n1)"
echo "Using package.json: $RNPKG"
jq -r '.dependencies // {} | keys[]' "$RNPKG" | sort > /tmp/deps.txt
printf "%s\n" fast-glob fuse.js ignore minimatch | sort > /tmp/need.txt
echo "Missing externals in dependencies (if you keep them external):"
comm -13 /tmp/deps.txt /tmp/need.txt || true

Length of output: 11152


Generate Node built-ins dynamically and bundle third-party modules
Replace the hardcoded built-in list in agent/runtime-node/build.ts with a programmatic builtinModules import (including node: prefixes) and only externalize VSCode API and the workspace package; bundle everything else (removing fast-glob, fuse.js, ignore, minimatch).

--- a/agent/runtime-node/build.ts
+++ b/agent/runtime-node/build.ts
@@ –1,4 +1,9 @@
-import { build } from 'esbuild';
+import { build } from 'esbuild';
+import { builtinModules } from 'node:module';
+
+const nodeBuiltins = Array.from(
+  new Set([...builtinModules, ...builtinModules.map((m) => `node:${m}`)])
+);

 export async function createBundle() {
   await build({
@@ –13,18 +18,7 @@ export async function createBundle() {
     // keep other options...
-    external: [
-      'vscode',
-      'os','path','fs','util','stream','events','buffer','crypto',
-      'child_process','http','https','url','querystring','assert',
-      'zlib','tty','net','dns',
-      'fast-glob','fuse.js','ignore','minimatch',
-      '@stagewise/agent-runtime-interface',
-    ],
+    external: ['vscode', ...nodeBuiltins, '@stagewise/agent-runtime-interface'],
   });
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
external: [
'vscode', // VSCode extension API should always be external
// Node.js built-in modules
'os',
'path',
'fs',
'util',
'stream',
'events',
'buffer',
'crypto',
'child_process',
'http',
'https',
'url',
'querystring',
'assert',
'zlib',
'tty',
'net',
'dns',
// Dependencies to prevent bundling issues
'fast-glob',
'fuse.js',
'ignore',
'minimatch',
'@stagewise/agent-runtime-interface',
],
import { build } from 'esbuild';
import { builtinModules } from 'node:module';
const nodeBuiltins = Array.from(
new Set([...builtinModules, ...builtinModules.map((m) => `node:${m}`)])
);
export async function createBundle() {
await build({
// keep other options...
external: ['vscode', ...nodeBuiltins, '@stagewise/agent-runtime-interface'],
});
}
🤖 Prompt for AI Agents
In agent/runtime-node/build.ts around lines 13 to 40, replace the hardcoded list
of Node built-ins and third-party externals with a programmatic approach: import
builtinModules from 'module' (i.e., const { builtinModules } = require('module')
or ES import) and construct the external array to include only 'vscode' and the
workspace package '@stagewise/agent-runtime-interface' plus all Node built-ins
both with and without the 'node:' prefix (deduplicated); remove 'fast-glob',
'fuse.js', 'ignore', and 'minimatch' from externals so they are bundled, and set
the rollup/webpack external option to that generated array.

sourcemap: false, // Disable source maps for security
minify: true, // Always minify for published packages
treeShaking: true,
Expand Down
2 changes: 2 additions & 0 deletions agent/runtime-node/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
"type-check": "tsc --noEmit"
},
"dependencies": {
"fast-glob": "^3.3.3",
"fuse.js": "^7.1.0",
"ignore": "^7.0.5",
"minimatch": "^9.0.5"
},
Expand Down
Loading