Move search api to Worker, and search indexes to R2#100
Conversation
|
|
@copilot how to fix this error: $ bun --bun ./tools/export-search-indexes.ts ./.output/search/latest.json Bun v1.3.5 (Linux x64) |
| options: { | ||
| api: withBasePath("/api/search"), | ||
| type: "static", | ||
| delayMs: 500, |
There was a problem hiding this comment.
The default was so low that when I typed "router" it ended end with ro, rout and router http calls, first two were not even cancelled... :D
During the docs build,
/api/search.jsonis prerendered and exported as the search index.The website workflow uploads that index to R2 under
search/<github-sha>.json, then deploys the docs Worker withSEARCH_INDEX_KEY=<github-sha>.At runtime, the docs app still calls
/api/search, but the docs Worker intercepts that request and proxies it through theSEARCH_APIservice binding to the newhive-search-apiWorker.The docs Worker forwards the active
SEARCH_INDEX_KEYasx-search-index-keyheader, and the Search API Worker loads the matching R2 object, creates the Fumadocs search server, and caches it in Worker memory.PR previews reuse the same Search API Worker but upload and reference their own SHA-specific index.
The generated search index is 3MB raw.
Current
mainships 25MB file with search related data, that is sent to every user as 6-7mb gzipped content.Slow 4G network =
26sto load the file.Now this file is smaller (3MB raw) and sent from R2 to Worker (and cached per cf location), so in case of cold starts (that I try to eliminate with perwarming the cf location), the network has no impact on search performance.