Skip to content

feat(store): support Bun runtime via bun:sqlite (vec0 + FTS5)#436

Open
Guihal wants to merge 2 commits into
TencentCloud:mainfrom
Guihal:feat/bun-runtime-support
Open

feat(store): support Bun runtime via bun:sqlite (vec0 + FTS5)#436
Guihal wants to merge 2 commits into
TencentCloud:mainfrom
Guihal:feat/bun-runtime-support

Conversation

@Guihal

@Guihal Guihal commented Jul 8, 2026

Copy link
Copy Markdown

Closes #454

Summary

The standalone TdaiGateway (and the read-local-memory diagnostic CLI) hard-wired Node's node:sqlite. Bun does not implement node:sqlite, so running the gateway under Bun (bun src/gateway/server.ts) booted in degraded mode: the vector store never initialised (No such built-in module: node:sqlite), disabling vector/FTS recall and dedup.

This adds runtime detection: under Bun the SQLite handle is opened via the built-in bun:sqlite; under Node behaviour is unchanged (node:sqlite). bun:sqlite's Database.loadExtension() loads sqlite-vec directly, so the Node-only enableLoadExtension(true) step is skipped under Bun.

Why

Bun is a common runtime for this kind of sidecar (and pi, which embeds TDAI via the gateway, runs on Bun). Making the gateway runtime-agnostic lets it run under either without code changes.

Changes

  • src/core/store/sqlite.tsopenSqliteHandle(path) picks bun:sqlite (globalThis.Bun) or node:sqlite; enableLoadExtension(true) guarded (Node-only). Same exec/prepare/run/all/get/close + Buffer blob surface — no query changes.
  • scripts/read-local-memory/read-local-memory.ts — same runtime detection for the diagnostic CLI.

Verification

  • bun src/gateway/server.tsGET /health{"status":"ok","stores":{"vectorStore":true,"embeddingService":false}}; boot log shows FTS5 tables initialized (l1_fts, l0_fts).
  • vec0 KNN + FTS5 bm25() confirmed working under bun:sqlite via direct probe (extension loaded with vec0.so, cosine MATCH query returns correct rows).
  • Node path unchanged: node --import tsx src/gateway/server.ts still status:ok, vectorStore:true.

Notes

  • embedding.provider still defaults to "none" (BM25 keyword recall). Semantic vector recall still requires a remote/embedding provider as before — orthogonal to this change.
  • No new dependencies.

VectorStore and the read-local-memory diagnostic CLI hard-wired node:sqlite,
which is unavailable in Bun — the gateway booted in degraded mode (no vector
store, no FTS5). Detect the runtime and open the database via bun:sqlite when
running under Bun. Bun's Database.loadExtension() loads sqlite-vec directly,
so the Node-only enableLoadExtension(true) step is skipped under Bun.

Verified: bun src/gateway/server.ts boots with vectorStore=true and FTS5
tables initialized; node path unchanged.
@YOMXXX

YOMXXX commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review note after local verification: this currently fails build, so I cannot approve it yet.

What passed:

  • COREPACK_ENABLE_AUTO_PIN=0 pnpm test (67/67 passed)

What failed:

  • COREPACK_ENABLE_AUTO_PIN=0 pnpm build
  • Failure: scripts/read-local-memory/read-local-memory.ts(373,33): error TS2554: Expected 1 arguments, but got 2.

The immediate cause appears to be the new requireNodeSqlite() return type in scripts/read-local-memory/read-local-memory.ts: it exposes DatabaseSync as new (path: string) => DatabaseSync, while openSqliteReadonly() still calls new DbSync(dbPath, { open: false }).

Please fix the constructor typing / Bun adapter shape so build:read-local-memory compiles, then this can be re-verified. Note: local Node is v22.14.0, below the package engine >=22.16.0; the engine warning is expected and not the cause of the failure.

@Maxwell-Code07

Copy link
Copy Markdown
Collaborator

Thanks for your attention! We will review this as soon as possible. Meanwhile, we suggest you link the corresponding issue related to this matter.

The earlier Bun shim narrowed requireNodeSqlite()'s return type to a 1-arg
constructor, breaking openSqliteReadonly()'s `new DbSync(dbPath, { open: false })`
(TS2554). Revert requireNodeSqlite to node-only and make openSqliteReadonly
runtime-aware instead: under Bun open natively with { readonly: true } (bun has
no lazy-open/.open()), under Node keep the {open:false} + .open() + query_only
pragma path. Fixes `pnpm build` (build:read-local-memory tsc) while keeping the
Bun path correct at runtime.
@Guihal

Guihal commented Jul 9, 2026

Copy link
Copy Markdown
Author

Thanks for the review! 🙏

Fixed in 97f5195. Root cause: my earlier shim narrowed requireNodeSqlite() to a 1-arg constructor, so new DbSync(dbPath, { open: false }) in openSqliteReadonly tripped TS2554.

Reverted requireNodeSqlite to node-only and moved the runtime detection into openSqliteReadonly instead — that is the cleaner spot since the lazy-open pattern is API-specific:

  • Node: unchanged ({ open: false } + .open() + PRAGMA query_only)
  • Bun: native new Database(path, { readonly: true }) (Bun has no lazy-open / .open())

pnpm build (build:plugin + all three script builds incl. build:read-local-memory) is green locally; pnpm test still 67/67.

@Guihal

Guihal commented Jul 9, 2026

Copy link
Copy Markdown
Author

As an aside for this Bun-support PR — memory footprint of the standalone gateway under each runtime (idle baseline, identical code path, fresh empty TDAI_DATA_DIR):

runtime version RSS (MB) peak VmHWM (MB)
node (--import tsx) v26.4.0 571.0 654.6
bun 1.3.14 225.7 507.6

Bun settles at ~2.5× lower resident memory than node on the same gateway (native extensions loaded in both: sqlite-vec, @node-rs/jieba, tcvdb-text; extraction LLM = glm-5.2, not invoked during the measurement). Bun/JSC also returns pages to the OS more eagerly — a long-running production bun instance sits at ~187 MB RSS (peak 555 MB) over its uptime, whereas node tends to hold onto what it peaked at.

Methodology: each instance booted against a fresh empty data dir; RSS = VmRSS and peak = VmHWM read from /proc/<pid>/status after /health returned status:ok + 3s settle; Linux x86-64. One run each — directional, not a benchmark.

Not a correctness claim, just context on why running this as a long-lived daemon under Bun is appealing.

@Guihal

Guihal commented Jul 9, 2026

Copy link
Copy Markdown
Author

Thanks! No existing issue covered this, so I opened #454 and linked it here (Closes #454). Happy to adjust the issue scope/title if you'd prefer it framed differently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support running the standalone gateway under the Bun runtime

3 participants