Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
204 lines (180 loc) · 11.3 KB

ARCHITECTURE.md

File metadata and controls

204 lines (180 loc) · 11.3 KB
Raw

How Paperlight works

Implementation notes for anyone reading or modifying the code. The README covers what the extension does; this covers how.

Install from source

Most people should just install from the Chrome Web Store. To run the checkout instead:

git clone https://github.com/kabir0st/paperlight.git
  1. Open chrome://extensions in Chrome
  2. Turn on Developer mode (top-right toggle)
  3. Click Load unpacked and pick the paperlight folder
  4. Pin Paperlight from the puzzle-piece menu so it is one click away

Reading PDFs saved on your computer? Open the extension's details page and switch on Allow access to file URLs.

Permissions, and why

Permission Why
storage Remember your theme, intensity, and voice
Host access (<all_urls>) PDFs live at unpredictable URLs, so the detector has to be allowed to load anywhere. It exits immediately on anything that is not a PDF
contextMenus The right-click Read aloud and Start from here items
tts The Robot voice
offscreen A hidden page that runs the Fluent voice and plays audio

Theming Chrome's PDF viewer

Chrome renders PDFs in an out-of-process viewer, which rules out most page-styling tricks. Blend-mode overlays, backdrop-filter, and SVG filter references cannot reach the viewer's pixels, and in current Chromium the viewer is not even an element in the wrapper document (its <body> is empty), with content scripts blocked from the inner plugin frame.

What does work, verified against the current out-of-process viewer, is applying plain CSS filter functions to the wrapper document's <html> element, inside which the viewer is composited.

  • The content script runs in every page and frame, but exits immediately unless the document is a PDF (document.contentType === 'application/pdf', or a PDF <embed> is present).
  • On a PDF tab it filters the wrapper document's <html> element. For a PDF <embed> inside a normal web page it filters just that element, so the surrounding site is untouched.
  • Chromium does not reliably apply manifest-declared content CSS to PDF wrapper documents, so the script injects its own <style> node and mirrors the filter as an inline style.
  • Each theme is a combination of sepia(), invert(), hue-rotate(), contrast(), and brightness(), with the intensity value baked into the numbers.
  • Because styling is only ever injected on PDF documents, regular websites are never touched.

Read-aloud

  • The Robot voice uses chrome.tts (your operating system's speech engine) straight from the service worker.
  • The Fluent voice runs in an offscreen document that spawns a small pool of Web Workers for inference. Extension pages share one renderer thread, so running models on it would freeze the popup; the workers keep everything responsive. Synthesis is transformers.js (ONNX Runtime WASM), streamed into the Web Audio API.
  • Chunks are cut incrementally and sized by buffer health. The first chunk of a session targets ~70 characters so the first audio arrives as fast as the model can produce anything; the target ramps up to 350 characters (the best prosody, comfortably under the model's 509-phoneme ceiling) once about 15 seconds of audio is buffered ahead. A chunk is never less than one full sentence — the cut runs past its target to the next sentence end rather than halting mid-thought, falling back to clause commas and word boundaries only for sentences too long to speak at all. Chunks span PDF page boundaries, so a sentence broken across pages reads as one. About 30 seconds of audio stays scheduled ahead as backpressure, with the audio still being synthesized counted against that target.
  • The worker pool scales with demand, not with the machine. One worker is always kept warm. Extras (up to 3, and never more than hardwareConcurrency - 2 — each WASM worker is single-threaded, one core apiece) are spawned only while the buffer is under ~10 seconds, enough text remains to repay the warm-up, and the weights are already cached. Idle extras are terminated after 45 seconds, since each holds its own ~200 MB copy of the model. Results can finish out of order; a committer schedules them into the audio timeline strictly in sequence.
  • Stopping cannot abort inference — ONNX Runtime has no cancel — so stop bumps a generation counter and drops stale results when they land. A worker still grinding through a chunk nobody wants is terminated and respawned only if a new reading actually needs its slot (weights reload from cache in seconds).
  • The activity card shows the words being spoken. The offscreen document keeps a schedule of what plays when, and ticks the HUD twice a second with the sentence under the play head, its timing, background generation activity (worker count, seconds buffered), and overall progress. The HUD interpolates between ticks on its own clock and moves a word highlight by character share of the chunk's duration — the model reports no word timestamps, so the marker is an estimate, good to about a word. These ticks bypass setStatus deliberately: at 2 Hz they would churn the badge and session storage for something only the reading tab renders. Highlighting on the PDF page itself is impossible: Chrome's viewer is an out-of-process plugin with no DOM the extension can reach.
  • Whole-PDF reading fetches the PDF bytes and extracts text page by page with pdf.js. For the Robot voice, pages stream back to the service worker as queued chrome.tts utterances.
  • Start from here reads from a selection to the end of the document. Chromium hands a context-menu selection over as bare text — no page number, no offsets, truncated at about a kilobyte — so the offscreen document has to search the extracted text for it. Selections under 5 words are refused with guidance instead of matched: that little text almost always appears earlier in the document too, and starting in the wrong place is worse than asking for a longer selection. The two sides do not agree character for character (the viewer's copy resolves ligatures, joins hyphenated line breaks, and spaces things differently), so matching runs on a folded form: NFKD-decomposed, lowercased, reduced to letters and digits, with an index map back to the original text. Progressively shorter prefixes are tried, and a word-aligned hit is preferred so a short selection like "The" does not land inside "theory", and a match whose first character carries the selection's exact case outranks one that only matches folded — selecting the paragraph opener "Recurrent" must not resolve to a mid-sentence "recurrent" on an earlier page. A match at the document's very first words (the title zone) is held back as a last resort within its match length: Chrome sometimes hands over less of the selection than was highlighted, and a lone "Attention" must not start the reading at this very paper's title. Text repeated with identical case across pages still resolves to the first copy, which is as far as a bare selection string can go — the offscreen console logs the received selection and where it resolved for diagnosing reports.
  • Model weights download from the Hugging Face Hub on first use and are stored in the browser's Cache API. Nothing is re-downloaded afterwards, and no text or audio ever leaves your machine. Kokoro's speakers are separate 0.5 MB style tensors, so switching speaker costs one small fetch rather than a model reload. Only the q8 export is ever loaded (model_quantized.onnx), so there is exactly one set of weights to download and track.
  • Pause suspends the offscreen AudioContext. That freezes the scheduled playback tail (and with it the buffer arithmetic), and the dispatch loop checks the paused flag directly, so synthesis stops too rather than racing ahead while you are paused. The Robot voice uses chrome.tts.pause().

In-page controls

  • The launcher, settings panel, and activity card are a content script rendering into a shadow root, so no page stylesheet can reach them.
  • Chromium applies a CSS filter to every descendant of the element it is set on, so an ordinary overlay would be inverted along with the page under the Dark theme. These render in the browser's top layer via the popover API, which is painted outside ancestor filter effects. Verified against Chrome's PDF viewer, where a top-layer element composites cleanly above the plugin.
  • The settings panel is not a second copy of the popup. It embeds popup.html itself in an iframe (hence web_accessible_resources), so the popup logic runs unchanged and the two surfaces can never drift apart. The framed page reports its own height by postMessage, since a content script cannot read across the extension-origin boundary.

All executable code (the transformers.js bundle, ONNX Runtime WASM, pdf.js) ships inside the extension, as Manifest V3 requires.

Project structure

paperlight/
├── manifest.json    # Manifest V3 definition
├── content.js       # Detects PDFs, builds and injects the theme filters
├── hud.js           # In-page launcher, settings panel, activity card
├── popup.html/css/js# Popup UI: themes, intensity, voice picker, voice options
├── background.js    # Defaults, context menu, robot voice, offscreen lifecycle,
│                    # status fan-out to popup + badge + HUD
├── offscreen.html   # Offscreen document hosting playback + PDF extraction
├── offscreen.js     # BUILT coordinator bundle (pdf.js, playback, worker mgmt)
├── tts-worker.js    # BUILT inference worker bundle (transformers.js + engines)
├── src/             # Sources for the built bundles
│   ├── offscreen-main.js    # Dispatch loop, in-order commit, playback,
│   │                        # pdf.js extraction, statuses
│   ├── tts-pool.js          # Worker pool: demand-driven scaling, retries,
│   │                        # generation tokens for cancellation
│   ├── tts-worker.js        # Inference worker: model download + synthesis
│   ├── kokoro-engine.js     # "Fluent" voice (Kokoro-82M)
│   ├── kokoro-voices.js     # Speaker list, shared by the engine and popup
│   ├── tts-common.js        # Asset caching + incremental chunk cutter
│   └── vendor/phonemize.js  # Vendored from kokoro-js (Apache-2.0)
├── vendor/          # ONNX Runtime WASM + pdf.js worker (copied by build.mjs)
├── build.mjs        # esbuild bundling script (npm run build)
└── images/          # Toolbar icons and logo

offscreen.js and tts-worker.js at the repo root are build outputs. Edit the sources under src/ and rebuild:

npm install
npm run build

Both bundles and vendor/ are committed, so a plain load-unpacked install works without Node installed.

Internal naming

Message types and DOM ids still use a gentle- prefix (gentle-ping, gentle-hud, gentle-panel-height, gentle-page-pdf-hud), left over from the extension's former name. They are an internal namespace only, never shown to users, and renaming them means changing senders and listeners across files in lockstep.