@typescript/vfs: Module resolution on createProgram breaks

[Matsuuu]

With a "by documentation" example of vfs setup, the program breaks when trying to create it.

Repro:

import ts, { CompilerOptions } from "typescript";
import fs from "fs";
import { createDefaultMapFromNodeModules, createSystem, createVirtualCompilerHost, } from "@typescript/vfs";
import path from "path";

const compilerOptions: CompilerOptions = {
    target: ts.ScriptTarget.ESNext,
}
const fsMap = createDefaultMapFromNodeModules(compilerOptions, ts);

fsMap.set("test.ts", 'console.log("Foo")')

const system = createSystem(fsMap);
const host = createVirtualCompilerHost(system, compilerOptions, ts);

const typeCheckProgram = ts.createProgram({
    rootNames: [...fsMap.keys()],
    options: compilerOptions,
    host: host.compilerHost
});

Expected behavior:

The Program gets created and is usable

Actual behavior:

The program creation crashes with the stacktrace

      while (pos >= 0 && pos < text.length) {
                                    ^

TypeError: Cannot read properties of undefined (reading 'length')
    at iterateCommentRanges (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:9417:37)
    at reduceEachLeadingCommentRange (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:9521:12)
    at getLeadingCommentRanges (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:9551:12)
    at getJSDocCommentRanges (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:14091:429)
    at addJSDocComment (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:28767:36)
    at parseSourceFileWorker (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:28741:34)
    at Object.parseSourceFile (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:28572:26)
    at Object.createSourceFile (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:27760:23)
    at Object.getSourceFile (C:\prog\github\tsc-checker-demoing\node_modules\@typescript\vfs\dist\vfs.cjs.development.js:483:53)
    at findSourceFileWorker (C:\prog\github\tsc-checker-demoing\node_modules\typescript\lib\typescript.js:117503:25)

Node.js v18.14.0

[Matsuuu]

I did some discussions in discord and walked through the problem with a debugger. The files that are missing from the system are as follows:

        "lib.es5.d.ts",
        "lib.decorators.d.ts",
        "lib.decorators.legacy.d.ts",
        "lib.dom.d.ts",
        "lib.webworker.importscripts.d.ts",
        "lib.scripthost.d.ts"

And I've deviced a small hotfix for the issue:

import ts, { CompilerOptions } from "typescript";
import fs from "fs";
import { createDefaultMapFromNodeModules, createSystem, createVirtualCompilerHost, } from "@typescript/vfs";
import path from "path";

const compilerOptions: CompilerOptions = {
    target: ts.ScriptTarget.ESNext,
}
const fsMap = createDefaultMapFromNodeModules(compilerOptions, ts);
applyHotfix(fsMap);
fsMap.set("test.ts", 'console.log("Foo")')

const system = createSystem(fsMap);
const host = createVirtualCompilerHost(system, compilerOptions, ts);

const typeCheckProgram = ts.createProgram({
    rootNames: [...fsMap.keys()],
    options: compilerOptions,
    host: host.compilerHost
});

console.log(typeCheckProgram.getSourceFile("test.ts"));

function applyHotfix(fsMap: Map<string, string>) {

    let hotfixFiles = [
        "lib.es5.d.ts",
        "lib.decorators.d.ts",
        "lib.decorators.legacy.d.ts",
        "lib.dom.d.ts",
        "lib.webworker.importscripts.d.ts",
        "lib.scripthost.d.ts"
    ];

    function loadTypescriptLibFile(filename: string) {
        const libDir = path.dirname(require.resolve("typescript"));
        return fs.readFileSync(path.resolve(libDir, filename), "utf-8");
    }

    hotfixFiles.forEach(f => {
        fsMap.set("/" + f, loadTypescriptLibFile(f));
    });

}

---

The VFS system seems to not include required files for the compiler to work. I also noticed that using the esnext lib in compileroptions breaks as the mapping of files in the vfs library doesn't match the actual type declaration files shipped with the latest Typescript version.

---

One fix for the type declaration file mismatch would be to query the tslib path in local node setups to enumerate what files are present and only using those instead of a hardcoded list. For web setups this ofcourse is not suitable but for node setups it would be.

[Matsuuu]

#2802 aims to fix some of the enumeration errors where some hardcoded libraries don't exist

[typescript-bot]

Hello! As per #2804, we are automatically closing all open issues. Please see #2804 for a description of what issues and PRs can be accepted going forward.