feat(web): on-device web search for chat - #290
Open
kfaracik wants to merge 57 commits into
Open
Conversation
…g refusal
The hard fallback ("say I don't know") made the model refuse any question the
context did not cover — including a general question asked mid-chat merely
because a document was attached. Soften the ladder, not the grounding: first
state that the sources (or search results) contain no information about the
question, and only then answer from own knowledge, explicitly marked as such.
The refusal sentence is phrased so looksLikeNoAnswer recognises it. The
previous web wording ("could not find it in the search results") slipped past
that detector, so a refusal after a search was still attributed to sources and
flagged result pages as used.
The scope rule stays absolute but now states its reason — an absent document's
text is genuinely not available to the model — and the grounding hint next to
the question drops its duplicate of that rule, which sat closest to the
question and would have overridden the new fallback. It keeps only the
referent anchor for "this file" / "it".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Before a search, planWebSearch turns the latest message into a JSON plan
{needs_search, intent, queries}: cheap heuristics send already-concise
keyword queries verbatim without touching the model, a gating prompt
skips search for small talk, and anaphoric follow-ups are resolved
against the last conversation turns. Parsing is defensive (think-block
strip, JSON lifted out of prose, per-query sanitising, sub-query cap)
and every failure path falls back to the verbatim message clamped to a
searchable length, so a planner fault can never block a search. An
unterminated <think> block is eaten whole rather than leaking
deliberation into a query or a parsed plan.
When a round fails, reformulateForCorrection climbs a free ladder first
(unused planned sub-query, then intent, then broaden by dropping the
trailing token), deduplicated case- and whitespace-insensitively against
everything already run. reformulateWithEvidence then lets the model read
capped excerpts of the pages the failed round DID find and build one new
query around a lead it spots there, in the topic language, never
translating proper names.
detectTopicLanguage reads that language off result hosts (ccTLDs plus
Wikipedia subdomains, mobile included, plurality of at least two,
English never reported since it is the pivot), and nativeTitleQuery
lifts a native-language page title as a ready-made next query.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enrichWebResults fetches full article text for the top search results
(the rest keep their SERP snippets) and marks each page usable only when
extraction produced real content: at least WEB_CONTENT_MIN_CHARS of text
and no anti-bot challenge markers.
looksLikeBotWall recognizes challenge pages ("verify you are human",
"checking your browser", "access denied", ...) by title always and by
body only when the text is short — a real article that merely mentions
those phrases stays in. Without this filter an aggressive anti-scraping
page (Cloudflare and friends) would feed its challenge boilerplate into
the model context; with it the page degrades to its honest snippet.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ating - memoize the favicon image source and reset its failed/progress state on host change, so a recycled row does not flash the previous site's icon - guard the collapse animation's onCollapsed behind a mountedRef so runOnJS cannot fire after the list unmounts - gate the generating/processing activity on isLastMessage so only the last bubble reflects the in-flight send Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Without injection, enrichment always performs a real HTTP GET via extractArticle, so runWebSearch cannot be exercised offline — a fixture-based eval run would silently hit the live web. Thread an ArticleFetcher through the runWebSearch input in the same style as provider/embeddings/generate; the default stays extractArticle, so production behaviour is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A second utility call while one is still generating hit the native ExecuTorch module mid-generation and threw code 104 "model is currently generating", poisoning every subsequent turn. Concurrent callers now get '' (the callers' existing fallback path) instead. A hard 8 s interrupt of slow utility generations was considered and dropped: slow devices will instead be told up front that search takes longer there (tracked separately). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The offscreen SERP WebView persisted cookies, localStorage and HTTP cache from every engine it visited for the lifetime of the app install. Incognito keeps the scrape session ephemeral; cookies still work within a single challenge flow, but nothing survives the WebView instance. Trade-off to validate on device: engine challenge clearance no longer persists across queries, so CAPTCHAs may reappear more often. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The overall abort (90 s cap or user interrupt) settled the search promise but let the expensive work run on detached: page fetches ran to their own 8 s timeouts, the embedding loop processed up to 40 chunks to completion, and the scrape WebView kept loading the SERP. The abort signal now reaches every stage: fetchHtml/extractArticle take an external signal wired into their AbortController, enrichWebResults threads it into each fetch and skips pages once aborted, retrieveWebPassages bails out of the embedding loop, and the scrape host resets the WebView to the idle page on abort/cancel. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three promptUtils tests encoded the web branch's ratio-based char budget (CHARS_PER_TOKEN), which main replaced with a finer per-codepoint model in #246. Under that model latin text is 0.25 tok/char against a 1/3 no-sample default, so ASCII now earns a *larger* budget than getPromptCharBudget(model) returns with no sample — the old equality no longer holds by design. - assert ASCII never drops below the default rather than matching it - measure the overflow test against the budget the assembled prompt actually gets, not the no-sample default - size the trimming filler at 2x budget, matching main's own convention, so it still forces a trim under the larger ASCII budget llmStore's retry test predated the web branch's inline auto-retry, which now reduces the prompt budget and regenerates once before surfacing the error. Exhaust that attempt so the test still exercises what it names: retryLastGeneration reusing the persisted user message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
On-device web search for chat: the model can ground answers in live web pages, fully on-device (scrape → fetch → extract → retrieve → rank → converge), with a per-chat Web toggle, an inline search-trace, and a Sources sheet.
Highlights
tsc --noEmit).