CNfirmed

Find and verify sources for Wikipedia {{citation needed}} claims.

CNfirmed locates every {{citation needed}}-family tag in a Wikipedia article and extracts the claim being cited, plus surrounding context. It then looks for a source in two stages:

On wiki, free. Citations this article already carries, and citations other language editions attach to the same fact. No model, no API key, no cost.
On the web, paid. Only for the claims stage 1 could not answer: an LLM with web search discovers candidates and judges whether each actually substantiates the specific claim — not just mentions the topic.

Results are returned ranked, each with a ready-to-paste Wikipedia cite template.

This repo ships two things:

A Wikipedia user script (userscript/cnfirmed.js) — runs entirely in the browser. The wiki-local stage needs no key at all. The web search is free with Tavily + an open-weight model while the CNfirmed Worker's shared Tavily credits last; after that, or for a model that searches the web itself, it talks directly to Anthropic, Google (Gemini), or OpenAI, or to Tavily through the Worker, using a key you store in your own browser's localStorage. The Internet Archive stage needs no key either: an open-weight model (Qwen 3.8 on Wikimedia's Lift Wing by default, or models on Hugging Face through the CNfirmed Worker; see Open-weight model) plans its searches and checks its passages, with the Verify API as the fallback.
A Node CLI (cnfirmed) — for running the same pipeline from the terminal. Web search uses the Anthropic API; checking a candidate against the claim uses the Verify API.

Why

Topical similarity isn't the same as substantiation. Existing citation-finders point at pages that sound related; CNfirmed asks the model to read the candidate and answer: "does this passage support this specific claim?"

And a lot of the time nobody needs to search at all. A tagged sentence often sits beside sourced text whose citation covers it too, and other Wikipedias — frequently stricter about inline citation — have already sourced the same fact. Those citations are free to find, were vetted by a human editor, and are exactly what a web search will not surface.

User script (recommended)

The user script is the primary way to use CNfirmed. It runs on the Wikipedia page, with no server of its own. Besides your model provider, it calls archive.org, the CNfirmed Cloudflare Worker (open-weight models for the Internet Archive stage, and the relay for the Tavily provider) and the Verify API; none of them needs a key from you.

The wiki-local stage works with no API key at all. The web search starts out free, too: the Tavily + open-weight model provider (the default) searches on the Worker's own Tavily key while its credits last. When they run out, the panel says so and asks for your own Tavily key, or for a switch to Claude, Gemini, or OpenAI, which search the web themselves. A key you add is kept in your browser's localStorage and only sent to the provider you chose.

Install

Copy userscript/cnfirmed.js to User:Yourname/cnfirmed.js on en.wikipedia.org.

Add to User:Yourname/common.js:

importScript('User:Yourname/cnfirmed.js');

Reload any article that has {{citation needed}} tags.

Need an article to test on? Category:All articles with unsourced statements lists every article on en.wikipedia with at least one {{citation needed}} tag.

Usage

A small badge appears next to every [citation needed] superscript. Its colour is the claim's status: green when a source supports it, blue when there are leads to check.
Click a badge to open that claim in a panel docked to the right edge of the page (the article moves over to make room; drag the panel's edge to resize it). Both free searches start at once: citations already on Wikipedia, and Internet Archive books, whose passages a model then checks against the claim. The web search starts by itself only when both find nothing and you have a key set; otherwise it waits for a click (free searches are shared, so they always wait for one).
The CNfirmed (N) link beside the page tabs (in the tools menu on other skins) opens the list of all the page's claims with their status. Find sources for all (free) runs both free searches for every claim; when they are done, a button offers a web search for the claims still left with nothing, and says how many searches that is. ‹ 1 of N › in the panel's header steps through the claims, and clicking the count opens the list.
Find sources for a claim in the Tools menu (on every article, tagged or not) is for text nobody has tagged. Select it in the article, a whole sentence or a fact within one (at least three words, within one paragraph or list item), and click Find sources in the panel. It becomes a claim like a tagged one, listed under "Picked by you", and its badge goes where its <ref> would: after the selection's closing punctuation and any footnotes already there. Nothing happens when you select text without starting this first, and a page without tags gets only the Tools item until it is clicked. Picked claims last until the page is reloaded. Text in an infobox, table, caption or heading cannot be picked.
Under the claim, Edit sentence and Remove sentence are for a claim that turns out to be wrong, or that nothing supports. Both open the section's source editor in a new tab: one with the tagged sentence selected, to correct or reword, the other with it already taken out (and a pre-filled edit summary). Nothing is saved until you review the diff and save it yourself. A removed sentence takes its own references with it; if one of them was a named reference used elsewhere, the editor says so.
The provider and API key are behind the panel's gear. A key is only needed for the web search once the free searches are used up, and is kept in localStorage.

Providers

Provider	Default model	Override (set on `window.…` before the script loads)
Claude	`claude-sonnet-5`	`cnfirmedModelClaude`
Gemini	`gemini-flash-latest`	`cnfirmedModelGemini`
OpenAI	`gpt-5-mini`	`cnfirmedModelOpenAI`
Tavily + open-weight model	the open-weight model's reader	`cnfirmedModelHf`

Claude, Gemini and OpenAI are invoked with their built-in web-search tool (Anthropic web_search, Google googleSearch+urlContext, OpenAI web_search) so source discovery and verification happen in a single round-trip per claim. Tavily + open-weight model searches with Tavily first, then has the open-weight model judge only what came back.

Tavily + open-weight model is the default for anyone who has not picked a provider or saved a Claude key. Without a Tavily key of your own, its searches go on the Worker's key (the Worker's TAVILY_API_KEY secret; it is never in the script, which is a public wiki page). When those credits are spent the Worker answers 402 free_credits_exhausted; the script remembers that for six hours and, on a later page, asks the Worker (GET /tavily) whether credits are back.

Open-weight model

The Tavily reader and the Internet Archive librarian run on open-weight models that need no key. Settings (the panel's gear) picks where:

Choice	Reader	Planner, books, check	Through
Qwen 3.8 on Lift Wing (default)	`llm-qwen38-27b`	`llm-qwen38-27b`	llm-router on Toolforge, `POST /liftwing` (Wikimedia's model hosting; free)
Hugging Face	`openai/gpt-oss-20b`	`zai-org/GLM-5.2`	the CNfirmed Worker, `POST /hf` (paid for by the Worker while its credits last)

The choice is kept in localStorage (cnfirmed-open-model). A model set on window (cnfirmedModelHf, cnfirmedArchive*Model) wins over the choice, and goes to the host that serves it: Lift Wing's model ids all start with llm-. Set window.cnfirmedLlmRouterUrl to use another llm-router host.

Output

For each claim, the panel shows one list of sources, best first, whatever search found it:

Supports the claim / Supports part of the claim: something read the source against the claim (an open-weight model, or the Verify API, for a book's passages; your model for a web page).
Lead, not checked: a citation an editor attached to a sentence like yours, in this article (already cited in this article, with that sentence) or on another language edition (cited on de.wikipedia, with the sentence there). Leads, not verdicts: read the source before you paste it.
Probably copied from Wikipedia: a web page whose quote repeats the article's sentence word for word (WP:CIRCULAR). Listed last, with a warning. Pages on sites anyone can upload to (Scribd and the like) carry a warning too, as does a web source the model judged of low reliability for the claim.

Sources checked and found not to state the claim are not listed, only counted. Click a source to open it: the passage or quote, any warning, and

Insert in editor, which opens the section's source editor in a new tab with the <ref> in place of the {{citation needed}} tag and a pre-filled edit summary linking to [[User:Alaexis/CNfirmed]]. For a picked claim, which has no tag, the <ref> goes after the selected text, past its punctuation and existing footnotes; the text is found in the section's wikitext by its words, and if it is not found exactly once (text from a template, or the page changed), the <ref> is copied to the clipboard instead. A picked claim nothing supports has Add {{cn}}, which puts a dated {{Citation needed}} there the same way. Review the diff and save. A same-article lead inserts as <ref name="existing" />, re-using the citation already on the page — the smallest possible edit.
Copy <ref>, which puts the same <ref> on the clipboard.

For a web source or an Internet Archive book, the <ref> is a fully filled citation template, made the way ProveIt and VisualEditor's Automatic tab make one. Opening the source asks the wiki's Citoid (/api/rest_v1/data/citation/mediawiki/…) about it: a book's ISBN, else the DOI in the link (doi.org/10.…, …/doi/full/10.…), else the page itself. The wiki's MediaWiki:Citoid-template-type-map.json picks the template for what Citoid found (a newspaper article → {{cite news}}, a journal article → {{cite journal}}), and that template's TemplateData citoid map says where each field goes: |last=/|first= per author, |date=, |work=/|journal=, |volume=, |issue=, |pages=, |doi=, |location=, |oclc=… The link the source was found at and |access-date= stay, as do an Archive scan's |url-access= and |via=; the rest of what we had fills only what Citoid left out. A wiki without those pages gets built-in copies of en.wikipedia's. A lookup that fails, or takes over 8 seconds, leaves the plain citation. Leads from Wikipedia citations are inserted as their editors wrote them. On the command line, find completes the sources found to support a claim and archive its leads, before printing.

Under the list, Searched says what each search (Wikipedia, books, web) found, or that it is still running, and holds the button for any search not yet run or failed.

Node CLI

A standalone command-line tool. Useful for batch runs or scripting; the user script doesn't depend on it. The web search runs against the Anthropic API; every candidate is then checked by the Verify API, which needs no key.

Install

npm install
npm run build

Set your API key:

export ANTHROPIC_API_KEY=sk-ant-...
# optional:
export CNFIRMED_MODEL=claude-sonnet-5          # web search model
export CNFIRMED_VERIFY_URL=http://localhost:8080  # Verify API base (default: https://citation-verifier.toolforge.org)

verify and wiki need no Anthropic key; find needs one only for the web search.

Usage

# End-to-end on a Wikipedia article: wiki-local sources first, web search for
# whatever is left.
node dist/cli/index.js find "Eiffel Tower"
node dist/cli/index.js find "https://en.wikipedia.org/wiki/Eiffel_Tower" --max-claims 3

# Just the free stage: what can be sourced from wiki alone. No API key needed.
node dist/cli/index.js wiki "Eiffel Tower"

# Books on the Internet Archive — open, or borrowable with a free account — whose
# text carries each claim.
# No API key. Prints how many books survive each step of the funnel.
node dist/cli/index.js archive "Eiffel Tower" --max-claims 5
node dist/cli/index.js archive "Eiffel Tower" --record fixtures-out/  # save raw responses
node dist/cli/index.js archive "Eiffel Tower" --full-text  # read open books' whole text too (slower)
node dist/cli/index.js archive "Eiffel Tower" --check      # and check every lead's passages (one model call a claim)
node dist/cli/index.js archive "Eiffel Tower" --no-librarian  # the word search alone, no model calls

# Just the verifier (the Verify API) — no API key needed.
node dist/cli/index.js verify \
  --claim "The Eiffel Tower was 300 metres tall when first built." \
  --source "https://www.britannica.com/topic/Eiffel-Tower-Paris-France"

# Just the extractor (no API calls; debugging).
node dist/cli/index.js extract "Eiffel Tower"

find flags for the two-stage pipeline:

Flag	Effect
`--sister-wikis <n>`	Language editions to mine (default 4; `0` uses only this article's own references).
`--no-wiki`	Skip the wiki-local stage entirely and go straight to web search.
`--wiki-only`	Never run the web search, whatever the wiki stage finds.
`--always-web`	Run the web search even when a wiki-local source already substantiates.

Add --json to any command for machine-readable output.

Architecture

userscript/
  cnfirmed.js          # self-contained user script (no backend)

src/
  core/                # framework-agnostic library, used by the CLI
    fetchArticle.ts    # Wikipedia API → wikitext + metadata
    extractClaims.ts   # wikitext → [{ claim, context, section, offset }]
    mediawiki.ts       # shared api.php client (wikitext, interlanguage links)
    wikitext.ts        # wikitext → prose, sentences, sections, wikilinks
    wikitextRefs.ts    # <ref> parsing → url, title, work, quote, reusable name
    relevance.ts       # deterministic scoring: token overlap + translation anchors
    wikiSources.ts     # stage 1 — citations already on wiki (no model)
    internetArchive.ts # Internet Archive client: full-text search, metadata, book text
    archiveSources.ts  # Internet Archive books whose text carries the claim
    archiveLibrarian.ts # the Archive stage's models: plan, named books, search inside, check
    findSources.ts     # stage 2 — Claude + web_search → candidate sources
    verifySource.ts    # (claim, source) → verdict, via the Verify API — separable
    formatCitation.ts  # source → {{cite web|...}} / {{cite news|...}}
    runArticle.ts      # orchestrator: stage 1, then stage 2 only if needed
    anthropic.ts       # shared client + prompt-caching helper
    prompts.ts         # disk-backed prompt loader
  cli/                 # thin Commander wrapper over core
  prompts/             # find_sources.md
  policy/              # WP:RSP unreliable-source blocklist

The user script is intentionally standalone: it inlines its own prompt, blocklist, citation formatter and wiki-local stage so it has no build step and no dependency on src/. The CLI continues to use the src/ library. test/userscriptWikiSources.test.ts loads the shipped user script with browser globals stubbed and asserts it finds the same candidates, with the same scores and ranking, as the TypeScript core — so the two copies cannot drift apart silently.

The verifier is separable (CLI)

verifySource(claim, sourceUrl) is exposed both as a library function and as the cnfirmed verify subcommand. The function contract is stable.

It does not call a model itself. It posts the claim and the source to the Verify API — POST /v1/verify from alex-o-748/citation-checker-script, hosted at https://citation-verifier.toolforge.org (override with CNFIRMED_VERIFY_URL). That service runs the same benchmarked prompt, parser and quote check as the citation-checker userscript, so CNfirmed does not maintain a second verifier prompt. It fetches source_url itself; pass { sourceText } to send text you already have as source_content instead.

What comes back:

verdict ∈ {SUPPORTED, PARTIALLY SUPPORTED, NOT SUPPORTED, SOURCE UNAVAILABLE}, confidence 0-100 (the API's support_score), comments.
quote — the API's verified_text: the part of the model's quote actually found in the source. The model's unverified quote is never passed on.
reliability is "n/a": the Verify API grades substantiation only. Candidates are still screened against the WP:RSP blocklist before they reach the verifier, and judging reliability for the claim is left to the editor.

The service allows 30 requests a minute across all callers. On a 429, verifySource waits out Retry-After (up to three times) rather than failing the candidate; a source the API cannot fetch (422) becomes a SOURCE UNAVAILABLE verdict.

A cite template and <ref> snippet are emitted for every candidate; the verdict, confidence and quote shown alongside are what a human editor uses to decide whether to paste it.

Sources already on Wikipedia (the free stage)

Before anything is billed, CNfirmed looks for a citation Wikimedia already holds. Two passes, both deterministic code with no model in the loop:

The article's own references. A tagged sentence usually sits beside sourced text, and the neighbouring citation often covers it too. Each existing <ref> is scored on proximity to the tag (same paragraph, same section, elsewhere) plus weighted token overlap with the reference's own title, publisher and quote=. Outside the claim's section, proximity counts for nothing and the reference has to earn its place on what it is actually about. A hit pastes as <ref name="existing" />, re-using the citation already on the page.

Other language editions. Their references are precisely the sources a web search will not surface. The corresponding sentence is located without a translation model, using anchors that survive translation:

Numbers and dates — normalised across numeral systems, so ١٨٨٩ and 1889 are the same anchor.
Proper nouns — folded for case and diacritics, and only used between wikis that share a script.
Wikilink targets — resolved to the counterpart title on the target wiki through interlanguage links. This is what makes a claim locatable on a wiki whose script shares nothing with ours.

A match on the exact sentence is the real signal; the same anchors elsewhere in the paragraph count for less. Both passes deduplicate by URL, drop blocklisted domains, and rank by match strength.

What comes out is evidence, not a verdict: a human editor cited that source for a sentence that looks like your claim. The CLI's find still verifies these leads with the model before ranking them alongside web results; cnfirmed wiki and the user script's free stage present them unverified, with the sentence and the matched anchors, and leave the judgment to the editor.

Deliberately out of scope: Wikidata statements and their references. This was built, shipped and then removed — it found nothing on real articles. See the plan doc for why, before building it again.

Books on the Internet Archive (experimental)

A second free stage, in both the user script (it runs whenever a claim is opened, alongside the Wikipedia search) and the CLI (cnfirmed archive). It looks for the claim in the OCR text of digitised books. A book counts if an editor can read the passage: an open one, or one in the lending library, which anyone with a free archive.org account can borrow.

It does what a reference librarian does, with open-weight models for the steps word matching cannot do: restate the claim, think of the books that would state it, search inside them for the telling term, and read the passage. The models are the open-weight model chosen in Settings (Qwen 3.8 on Lift Wing by default; the GLM-5.2 below is the Hugging Face choice), so the editor needs no key. Why each step is there, and what each one found on the evaluation set, is in docs/internet-archive-search-ideas.md.

Plan — a model (GLM-5.2) restates the claim with the names its paragraph gives ("Feodora's father died in 1814" → "Emich Karl, Prince of Leiningen, the father of Princess Feodora, died in 1814"), writes three full-text queries in a book's wording rather than Wikipedia's, and lists what a passage stating the claim must mention, numbers and names first.
Search — the planner's queries, beside the word search's own two families, run side by side, each strictest first until it has ten books:
- numbers and names: the article's subject with every number and name in the claim, then with the numbers only, then with the strongest anchor;
- key phrases: the subject's words with the claim's key phrases — runs of words between punctuation, function words and common verbs, a longer run split into pairs ("digestive enzymes", "father died", "Ptolemy II Philadelphus") — then fewer phrases, and last two phrases without the subject, since a Wikipedia title is often not how a book names its subject.
Terms are joined by a space, which the endpoint reads as AND. An explicit AND is not safe: between two terms it is taken for the word "and" ("Feodora" AND "Leiningen" returns exactly what "Feodora" "Leiningen" "and" does), and each book's few passages are then spent highlighting "and". A claim with neither a number, a name nor a key phrase gets no word search.
Name books — a model (GLM-5.2) names up to six books likely to state the fact: biographies, histories, monographs, reference works. Each is looked up in the Archive's catalogue by its title's words and its author's surname; up to two readable scans of each are kept, since the first scan is not always in the full-text index. Small models invent titles instead; see the doc above.
Access gate — on each hit's own fields: lending-library books (inlibrary) are kept as borrowable; books only certified print-disabled readers can open (printdisabled without inlibrary) are dropped; everything else is open.
Score — each hit arrives with its matching passages, and each is scored on its own (two passages from one book may be pages apart):
- the word search's hits with the sister-wiki scoring (a claim number and the subject, then anchors and weighted token coverage) and by claim units (two of the claim's phrases, numbers, names or longer words, one of them a phrase, number or name, and a word of the subject unless the book's title is about it). The two rankings are taken in turn, four books kept;
- the planner's hits by how many of the plan's concepts their highlights mention. A book must be about the claim: one passage with two concepts, and one of the plan's names when it has any. Without that, "New York, 1814 DEATH (the) of Abel" was a lead for Feodora's father. Up to five are added.
Dedupe — one book per work across all three sources: the same main title and a creator sharing a name. After the metadata lookup, two leads sharing a passage word for word are one text uploaded twice, and are shown once.
Search inside — every lead is searched inside for the claim's three most telling terms (the year, the name), one full-text query per term over the whole shortlist: "1814" identifier:("a" OR "b" …) returns that term's highlights in every book on it, borrowable ones included, whose text is not otherwise public. Each identifier is quoted, and one with a dot (the Digital Library of India's in.ernet.dli.2015.…) is left out: identifier: never matches it, and one left unquoted empties the whole query. A named book with no passage is dropped; its first scan with one stands for it.
Look up — item metadata for the leads kept (the word search's four, then up to eight more): publisher and ISBN for the citation, whether the book has been withdrawn, and whether the item bundles several books. Some uploads put a whole series in one item under the first book's title ("009 Bernard Montgomery" holds four Osprey books, "018 - Eisenhower" among them), and its page opens on that first book. A hit says which file matched, so a lead in such an item takes that book's name, links to /details/{id}/{book}, and carries a warning to check the book's title, author and publisher; the item's own author, year, publisher and ISBN are left out of its citation. A search inside such an item keeps only the lead's own book's passages.
Whole text (off) — with --full-text (CLI) or window.cnfirmedArchiveFullText = true (user script), the whole OCR text of up to five open books is read for the best ~450-character stretch. archive.org sends no CORS header on that page, so a browser refuses it unless the switch names a proxy prefix.
Check — every lead's passages go to a model (GLM-5.2) in one call, with the claim, its paragraph and the restated claim. Each book gets a verdict — states the claim, states part of it, does not state it — with the words from its passages that state it and a short reason. A verdict counts only when those words are in the passages (allowing OCR line breaks and ellipses); otherwise the book counts as not stating the claim. On the hand-labelled leads this check kept 12 of 19 good leads and passed 2 of 77 bad ones (eval/experiments/calibrate-check.ts). On the 100-claim evaluation, the leads it keeps are 55% good, against 18% for the word search's leads before the librarian (eval/README.md). The panel lists the books that state the claim or part of it, and only counts the rest. If the model call fails, each lead goes to the Verify API instead (POST /v1/verify at citation-verifier.toolforge.org, one call per book, the restated claim as the claim); if that fails too, the leads are shown marked not checked, with a button to try again. The CLI runs it with --check.

Each step can be changed or turned off before the script loads: window.cnfirmedArchivePlannerModel, window.cnfirmedArchiveBooksModel, window.cnfirmedArchiveJudgeModel (a model on the Worker's or llm-router's allowlist, or false; "openai/gpt-oss-20b" as the planner costs next to nothing and finds nearly as much), and window.cnfirmedArchiveLibrarian = false for the word search alone (the check keeps its own switch). In the CLI, --no-librarian. Set window.cnfirmedVerifyUrl or window.cnfirmedProxyUrl (CNFIRMED_PROXY_URL for the CLI) to use other hosts.

That is up to seven word-search queries, three planner queries and three search-inside queries per claim (the full-text search takes some 25 seconds a query, run side by side), up to six catalogue lookups and a dozen metadata requests, and three model calls (plan, books, check): about 3 US cents a claim, paid by the Worker. The Worker allows 20 requests a minute per address, and the script keeps under 18. The Worker gives up on a model after 60 seconds, and now and then GLM-5.2 thinks until it runs out of tokens with no answer: each call is capped at 6,000–7,000 tokens (the books call asks for low reasoning effort), and a call that gets a 5xx or no answer is tried once more. Each lead comes with its passages, the anchors or concepts they matched, a link that opens the book searched for the passage's own words — three to six consecutive words, quoted, the run with the most of the claim's terms and then the most content words ("promoted to lieutenant-general on July 7"), never across an ellipsis or a word broken across OCR lines, and taken from the checked quote once there is one. The passage is the page's own OCR text, so any run of it finds the page; a single common word such as "July" or "Pantheon" matched a dozen pages — and a {{cite book … |via=Internet Archive}} — with |url-access=registration for a borrowable book, as InternetArchiveBot writes it. The page number is not known: the editor adds |page= after checking the passage. Old sources can be outdated or primary (WP:AGEMATTERS).

Endpoints. Only https://archive.org is on Wikipedia's CSP allowlist, so both front ends use the full-text search archive.org's own search page uses (/services/search/beta/page_production/?service_backend=fts), the catalogue search (/advancedsearch.php) and /metadata/{id}; all three were checked from a Wikipedia page. The ia package's be-api.us.archive.org, search-inside-the-book and OCR downloads are refused there, and are not used anywhere, so the CLI and the user script see the same data. The response shape is recorded in test/fixtures/archive/. Checked live: a space is AND; an exact "phrase" works and is highlighted as one passage; identifier:x, identifier:("a" OR "b") (not for identifiers with a dot) and field ranges such as year:[1800 TO 1930] work. AND between two terms is read as the word "and"; OR, NEAR and proximity ("a b"~20, which reads 20 as a term) do not work. A / (even inside quotes, where it starts a regular expression), ~, [ or ], or an unbalanced quote fails the whole query with HTTP 400, so they are taken out of every phrase and of the planner's queries. Now and then the backend fails a query it accepts (HTTP 400, "the search backend encountered an exception"), and archive.org then serves that failure for the same URL for some 20 minutes: both front ends ask twice more, each time for one more hit (a fresh search), after 5 and then 15 seconds, since the failures come in bursts. Each book comes back with at most five highlights of about 100 characters, each around one term.

Periodicals. The search surfaces many digitised magazines and journal volumes (Scientific American, Engineering), which are good sources, but every page of an 1889 issue prints "1889" in its masthead. For an item in the periodicals collection, its own year is therefore not counted as a matched number: a claim whose only number is that year gets nothing from that issue. Without this rule, an Engineering index line — "JULY 19, 1889 … Electricity on the Eiffel Tower, 702, 703" — scored 0.85 for "The tower opened to visitors in 1889", higher than any genuine passage.

Measuring it. The funnel line — 4 queries → 41 books → 38 readable (29 to borrow) (dropped: 3 print-disabled readers only) → 4 with a matching passage, 3 by claim phrases → 4 lead(s) — is shown when you hover over the panel's Books line, logged to the browser console, and printed by the CLI, which can also save the raw responses with --record <dir>. It is not part of the CLI's find until those numbers say it earns its place.

Policy handling (three layers)

Hard domain blocklist — src/policy/unreliable_sources.ts for the CLI, mirrored inline in userscript/cnfirmed.js. Catches WP:RSP-deprecated outlets deterministically, and is applied to wiki-local candidates too (so a wikipedia.org URL cited on another wiki never comes back — WP:CIRCULAR).
Discovery-time prompt guidance — src/prompts/find_sources.md (CLI) and the inlined system prompt (user script) steer the model toward WP:RS-compliant sources during search.
Verifier reliability axis — the context-sensitive judgment the domain blocklist cannot express. The user script's web search still grades it; the CLI's verifier (the Verify API) does not, so there it is the editor's call.

Testing

npm test                                  # everything
npx tsx --test test/wikiSources.test.ts   # one file
npm run eval                              # the free stages on 100 real {{cn}} claims

npm run eval measures what the tests cannot: the shipped user script, run over 100 real {{citation needed}} claims drawn from Category:All articles with unsourced statements, replayed offline from recorded responses in a few seconds, and scored against hand-checked answers. See eval/README.md.

The unit tests are fixture-based and fully offline — no network, no API key. Coverage: the wikitext claim extractor, <ref> parsing (named refs, list-defined refs, identifiers, archive fallback), the relevance scoring, the two wiki-local passes end to end (including a cross-script Japanese fixture), and the user-script parity test. Add fixtures in test/fixtures/ as edge cases appear.

Status

v1 ships the user script + the CLI + core library, with the wiki-local stage in front of the web search in both. Deferred:

Wikidata statements and their references
Browser extension wrapper (no install via common.js)
MCP server wrapper
Bot / talk-page integration
Non-Wikipedia wikis

See docs/source-quality-and-cost-plan.md for the wider plan on source quality and making the tool free to run without an API key. The wiki-local stage is phase 2 of that plan.

Both passes have been exercised against the live API from a user script on en.wikipedia.org. The page's CSP is an allowlist: every Wikimedia project, Toolforge, the three model providers, archive.org and doi.org are on it, but most third-party APIs are not, and a request to one is refused before it leaves the browser. A new retrieval source has to be checked against that list first (see the plan doc); what is not on it needs a Toolforge proxy.

License

MIT — see LICENSE. The copy published on Wikipedia is also available under the wiki's own terms (CC BY-SA 4.0 and GFDL).

Navigation Menu

Repository navigation

About

Repository files navigation

CNfirmed

Why

User script (recommended)

Install

Usage

Providers

Open-weight model

Output

Node CLI

Install

Usage

Architecture

The verifier is separable (CLI)

Sources already on Wikipedia (the free stage)

Books on the Internet Archive (experimental)

Policy handling (three layers)

Testing

Status

License

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
docs		docs
eval		eval
src		src
test		test
userscript		userscript
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
tsconfig.build.json		tsconfig.build.json
tsconfig.json		tsconfig.json

Latest commit

History

Folders and files

Repository files navigation

CNfirmed

Why

User script (recommended)

Install

Usage

Providers

Open-weight model

Output

Node CLI

Install

Usage

Architecture

The verifier is separable (CLI)

Sources already on Wikipedia (the free stage)

Books on the Internet Archive (experimental)

Policy handling (three layers)

Testing

Status

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages