Skip to content

AGENTS.md

Latest commit

 

History

History
334 lines (276 loc) · 22.9 KB

AGENTS.md

File metadata and controls

334 lines (276 loc) · 22.9 KB
Raw

AGENTS.md — Dollchan Extension Tools

Userscript + browser extension that adds features to imageboards (4chan, 2ch.hk, and engines like Wakaba/Kusaba/Tinyboard/Vichan/TinyIB/LynxChan/FoolFuuka).

  • ESNext userscript: src/Dollchan_Extension_Tools.es6.user.js
  • ES5 userscript: Dollchan_Extension_Tools.user.js
  • Extensions: extension/v2 (MV2), extension/v3 (MV3)
  • Source of truth: src/modules/*.js — every file listed above is generated.

Golden rule: never hand-edit generated files

.gitattributes marks them linguist-generated=true, so diffs are hidden and reviewers see nothing.

Generated file Produced by
src/Dollchan_Extension_Tools.es6.user.js gulp make:es6 (concat of src/modules/* + header from .meta.js)
Dollchan_Extension_Tools.user.js gulp make:es5 (browserify + babelify, comments stripped)
extension/v2/Dollchan_Extension_Tools.es6.user.js gulp copyext (<EXCLUDED_FROM_EXTENSION> blocks removed)
extension/v3/Dollchan_Extension_Tools.es6.user.js same as above

Always edit src/modules/*.js (or src/es5-polyfills.js, Dollchan_Extension_Tools.meta.js) and rebuild.

The artifacts are committed exactly as the local build writes them: make:es6 normalizes them to LF, so the bundle is byte-identical whether the checkout is LF (Linux) or CRLF (Windows, upstream's case). Upstream instead writes \r\n before the body (commit 668eebe4, "Add new line"), which on a Linux checkout left a single CRLF among LF breaks. Their diffs are binary-marked in .gitattributes, so this churn is invisible in review — do not hand-normalize them, the next make undoes it. Compare them with diff <(git show HEAD:<file> | tr -d '\r') <(tr -d '\r' < <file>) to see the real change.

Build

node_modules is not committed. First run npm install, then use the local gulp CLI (npx gulp <task> if there is no global gulp).

Task Effect
gulp make full build: make:es5 (which itself chains make:es6)
gulp make:es6 ESNext bundle only — does not refresh extension/
gulp make:es5 make:es6 + ES5 bundle (browserify/babelify) + copyext
gulp copyext copies the ESNext bundle into extension/v2, extension/v3 (strips <EXCLUDED_FROM_EXTENSION> blocks)
gulp make:modules Destructive: splits the built ESNext file back into src/modules/*
gulp default make + watch on src/modules/*, src/es5-polyfills.js, *.meta.js
gulp updatecommit Rewrites const commit = '…' in Wrap.js from git rev-parse HEAD
gulp bump Increments the version in every file that carries it — run before make

updatecommit is the first step of make:es6 (and thus of make:es5 and make), so a build always leaves src/modules/Wrap.js modified. That is expected — do not revert it silently.

Version bumps

Bump the version on every commit. Userscript managers only offer an update when @version grows, so a commit that ships without a bump is a fix nobody receives. The scheme is a counter in the fourth part: 24.9.16.0 → 24.9.16.1 → 24.9.16.2 (the first three parts stay as the upstream release this fork is based on).

The exception is a commit that changes nothing the build consumes — AGENTS.md, tools/, README.markdown. There is nothing for a user to receive, and a bump would offer every install an update that changes nothing, so such commits are made as they are. Bump whenever something under src/, extension/ or Dollchan_Extension_Tools.meta.js changes: that is what the published builds are made of.

npx gulp bump     # 24.9.16.N -> 24.9.16.N+1, in every file that carries it
npx gulp make     # then rebuild: the artifacts bake in the version from meta.js and menu.html

gulp bump rewrites the six files that carry the full 4-part version — src/modules/Wrap.js, Dollchan_Extension_Tools.meta.js, both extension/v*/manifest.json, both extension/v*/menu/menu.html — and fails loudly if one of them does not contain the current version. Order matters: bump first, build second, never the other way round.

package.json and package-lock.json are deliberately not bumped: npm rejects 4-part versions, so they keep the 3-part 24.9.16 form and only change if the first three parts ever do. Both Chrome and Firefox accept 4-part extension versions (verified by loading extension/v3).

The update path must point at this fork

@updateURL in Dollchan_Extension_Tools.meta.js and gitRaw in GlobalVars.js are what make updates work, and both must name this fork (acidnik/Dollchan-Extension-Tools) — inherited upstream values make managers check the upstream script and silently offer its build. gitRaw is also what Misc.js checkForUpdates() scrapes for the remote const version/const commit, so it has to be this repository or the in-app check answers about the wrong project.

Never change @namespace, @name or @description: managers match an installed script by namespace and name, so changing either makes it a different script and the existing installs stop updating. gitWiki intentionally stays on the upstream wiki — this fork has no wiki of its own, and those pages document the same features.

Module system (important)

src/modules/Wrap.js is the template: it opens the single IIFE and holds /* ==[ Name.js ]== */ placeholders. gulp make:es6 replaces each placeholder with that file's contents, in the order the placeholders appear; the last marker (/* ==[ Tail ]== */) closes the IIFE and stays inline in Wrap.js.

Consequences when writing code:

  • All ~38 modules are concatenated into one lexical scope. There is no module isolation.
  • Do not redeclare a top-level name that already exists in another module — use distinct names or reuse.
  • Order in Wrap.js is execution order. Top-level const/let are subject to TDZ; function declarations are hoisted, which is why most cross-module calls (e.g. runMain → DelForm) look unordered.
  • Shared mutable globals (aib, Cfg, doc, lang, nav, pByEl, …) are declared in GlobalVars.js by design.
  • A new module file is ignored by the build until you add its /* ==[ Name.js ]== */ placeholder to Wrap.js in the right position.

Code style

Style is defined by eslint.config.mjs (flat config; ESLint 10). There is no CI lint job and no test suite.

  • 4 spaces for indentation, tabs are banned (no-tabs); max-len 120; no trailing whitespace; semi always; single quotes.
  • No space after if, for, while, switch, catch → if(x) {, for(let i = 0; …), catch(err) {.
  • Never a space before the parenthesis of named/anonymous functions (function foo(a) {), but always for async arrows: async (a) => {}.
  • comma-dangle: never, object-curly-spacing: always, array-bracket-spacing: never.
  • Object literals: as-needed key quoting, and colons aligned in multi-line literals (someKey : value). object-shorthand: always.
  • one-var: consecutive uninitialized declarations are grouped (let a, b, c;); initialized ones never are.
  • prefer-destructuring (objects), prefer-const, no-extra-parens, empty catch {} allowed.
  • Naming and prefixes are load-bearing: $-prefixed DOM helpers ($q, $id, $bEnd, $popup), de--prefixed CSS/classes/attributes (de-post, de-cfg-tab, [de-form]), Cfg for user settings, nav for browser capabilities (from initBrowser() in Browser.js).
  • .eslintrc.json inside src/modules/ disables no-undef/no-unused-vars for the concatenated scope. Be aware flat config does not read eslintrc files, so npx eslint on this repo reports many pre-existing errors (no-undef, no-unused-vars): treat lint as a style reference, not a clean gate, and match the surrounding code. linebreak-style is unix here: the repository, the build and the served bundle are LF only.

Localization

All user-facing strings live in Lng in src/modules/Localization.js as [ru, en, ua] triples, selected by lang (0 = ru, 1 = en, 2 = ua from Cfg.language).

  • Settings labels go under Lng.cfg.<settingId> (a triple of strings, or { sel: [...], txt: [...] } for dropdowns).
  • Every new string needs all three languages; never leave '' unless the upstream does (see language).
  • Comments in Localization.js mark which settings tab each block belongs to.

Adding a setting

  1. src/modules/DefaultCfg.js — add the default value to defaultCfg (with an inline comment).
  2. src/modules/Localization.js — add the label under Lng.cfg (string triple, or { sel, txt } for dropdowns).
  3. src/modules/WindowSettings.js — render the control inside the matching _getCfg*() tab builder: _getBox(id, needReload) for checkboxes, _getInp(id, addText, size) for text/number fields, _getSel(id) for dropdowns. Controls re-read Cfg by their info attribute, so it must exactly match the defaultCfg key.
  4. Register the option in _updateDependant() if it gates other controls, and mark reload-required ones with de-cfg-needreload / the needReload argument.
  5. CfgWindow.handleEvent — add a switch(info) case if the option must take effect immediately (re-render CSS, re-init a subsystem, update posts); otherwise it applies on the next page load.
  6. Read it as Cfg.<id>; persist with CfgSaver.save('<id>', value), or toggleCfg(id) for checkboxes (per-domain storage in Storage.js, loaded by readCfg()).

Module map

Area Files
Entry / globals Wrap.js, Main.js, GlobalVars.js, Misc.js, Browser.js
Config & i18n DefaultCfg.js, Localization.js, Storage.js
Board support BoardDetector.js (getImageBoard), BoardDefaults.js
UI shells Panel.js, WindowUtils.js, WindowSettings.js, WindowFavorites.js, WindowVidHid.js, MenuPopups.js, Hotkeys.js
Posting & form Form.js, FormSubmit.js, FormFile.js, FormCaptcha.js
Posts & media Posts.js, PostBuilders.js, PostPreviews.js, PostImages.js, RefMap.js, Players.js, ContentLoad.js
Threads & page Threads.js, ThreadUpdater.js, DelForm.js, Pages.js, TimeCorrection.js, Ajax.js
Filtering & styling Spells.js, Css.js, SvgIcons.js, Utils.js

Finding your way around (debugging)

Locate behaviour before reading whole modules — the codebase is ~40k lines with no tests.

  • From an error message to its source. User-visible text lives in Lng (Localization.js) and is often concatenated at the call site (Lng.cantLoad[lang] + ' URL: ' + url), so grepping the sentence you see finds nothing. Grep a short fragment, or better the key name (cantLoad), then inspect its callers. Popups are [id^="de-popup-"] ($popup(id, msg)), so a popup id is also a grep target.
  • All network goes through one function. $ajax(url, params, isCORS) in Ajax.js is the only primitive; most callers reach it via ContentLoader.loadFileData() or ajaxLoad()/AjaxCache. The third argument is not "is cross-origin" in practice — callers pass !url.startsWith('blob') to keep blob URLs off the CORS paths. Branch selection: fetch when nav.canUseFetch && ((isCORS ? !nav.hasGMXHR : !nav.canUseNativeXHR) || aib.hasRefererErr), otherwise GM_xmlhttpRequest when (isCORS || !nav.canUseNativeXHR) && nav.hasGMXHR, otherwise native XMLHttpRequest. Careful: one failed xhr.open() permanently sets nav.canUseNativeXHR = false, after which everything silently switches to fetch.
  • "Why does it differ on board X?" means one of two flag sets. Per board — aib.* from BoardDetector.js: find ibDomains['domain'] = Class, then walk class X extends Engine up to BaseBoard (BoardDefaults.js), which holds the defaults and getters; engines set their discriminating flags in constructors (hasRefererErr, multiFile, jsonSubmit, noMarkupBtns, qForm, qPostImg, …). Board quirks are almost always these flags plus the site's own HTML and CSP. Per environment — nav.* from initNavFuncs() in Browser.js: scriptHandler (Tampermonkey/Violentmonkey/WebExtension/In-page), isFirefox/isWebkit/isMobile, canUseFetch, hasGMXHR, hasWorker. Userscript-manager and extension differences live here, so a bug that reproduces in only one manager is a nav branch.
  • Runtime objects (declared in GlobalVars.js, assigned in Main.js): aib = board descriptor, nav = environment, Cfg = per-domain user settings, lang = 0/1/2, postform = the single PostForm, DelForm.first / Thread.first / pByNum / pByEl = parsed page. Event handlers are handleEvent(e) objects registered on elements, so grep the event type ('paste', 'drop') to find them.

Reproducing a board bug in a real browser

Do not reason a board bug out of the source: engine flags, the site's CSP and its live HTML decide the outcome. Drive the real page headlessly instead — minutes instead of an hour of grep. The scripts live in tools/ (tools/README.md documents them and the env vars):

  • node tools/repro-file-paste.mjs — injects the built userscript into a live board, dispatches Ctrl+V or drag&drop into the reply form, and reports the form state plus pageErrors and requestFailures. Exits non-zero when the form rejects the file, so it doubles as a check. ACTION=drop is the baseline to compare against: when both paths fail identically, the bug is not in the path you just changed.
  • node tools/probe-blob-csp.mjs — the minimal-probe pattern: isolate one mechanism (transport × CSP × execution world) in a few lines instead of reasoning about it. Use this shape whenever a request fails oddly, before changing app code.
  • Both resolve playwright and a cached Chromium themselves, and both need npx gulp make first because they load the generated bundle, not src/modules/*.
  • The injected bundle runs in the main world without GM_*/chrome.storage, so Dollchan falls back to localStorage and scriptHandler: 'In-page' (Browser.js). That covers UI/form/post bugs but never privileged paths (GM_xmlhttpRequest, chrome.runtime).
  • With a board CSP, which build is affected depends on the kind of request. fetch/XMLHttpRequest initiated by a content script are not subject to the page's CSP — that is why the Ctrl+V blob bug hit the userscript only. A subresource of an element living in the page document (<img>/<video> with a blob: src) is checked against the document's CSP and is blocked in the extension too: endchan.org blocks video previews in both builds that way. Do not assume "extension = CSP-safe".
  • Assert a language-independent postcondition (a class, an input value, a popup id): readCfg() picks the language from navigator.language, so text assertions depend on the context locale.
  • Always collect console + pageerror + requestfailed: CSP violations exist only there, as requestfailed <url> :: csp plus a console.error naming the violated directive. That is how the Ctrl+V blob bug was identified.

Testing (manual)

There is no automated test suite; verification is manual in a real browser.

  • After editing modules, run gulp make before testing — extension/ and the root userscript are stale otherwise.
  • Chrome: chrome://extensions/ → Developer mode → Load unpacked → extension/v3.
  • Firefox: about:debugging#addons → Load Temporary Add-on… → extension/v3/manifest.json.
  • Firefox for Android: npm run start:firefox (= web-ext run --source-dir ./extension/v2) with a device attached.
  • Verify against a real imageboard; note that aib is per-engine (BoardDetector.js) and most code paths are engine-dependent, so a fix for one board may need parallel handling elsewhere.

Pitfalls learned from bugs

  • Never re-read a File/Blob you already hold through URL.createObjectURL + $ajax/loadFileData. Boards can send a CSP whose default-src (i.e. the connect-src fallback) does not allow blob:, which makes that request fail with status 0 — e.g. endchan.org breaks Ctrl+V image paste this way. Use readFile() / FileInput._readDroppedFile() instead; clipboard, drag&drop and file-picker files should all follow the same direct-read path. Note that a board's img-src may equally reject blob:, in which case preview thumbnails stay blank even though the file is attached correctly.

Git

Do not stage, commit, or push unless the current user message explicitly asks for it ("commit", "push", "deploy"/"деплой"). gulp make dirtying Wrap.js and regenerating the artifacts is not permission to commit.

The repository is LF only and indents with 4 spaces; upstream is CRLF with tabs, and that difference is what tools/sync-upstream.mjs exists to absorb — it lands upstream's tree in this fork's convention, then merges with -X ignore-space-change. After any upstream merge the bundle must still lint clean: npx eslint src/Dollchan_Extension_Tools.es6.user.js is what catches upstream's tabs coming back.

Starting a fresh session here

  • Check the working tree first. A previous session can leave work uncommitted: git status --short, then read the diff — the code alone will not tell you what is half-finished (the alternative reply form layout was left like that). If source and artifacts disagree, npx gulp make settles it: it is cheap and idempotent.
  • After pushing, verify the update path from outside: fetch the raw Dollchan_Extension_Tools.meta.js and src/modules/Wrap.js from raw.githubusercontent.com/acidnik/Dollchan-Extension-Tools/master/ and check @version, const version and const commit. Userscripts only update from what is on master.
  • To let the user try an uncommitted build: npx gulp bump && npx gulp make, then they open src/Dollchan_Extension_Tools.es6.user.js in the browser (or reload the unpacked extension/v3). Without a bump their manager will not offer it — the version is what triggers an update.
  • Screenshots and other throwaway artifacts for the user go to tmp/ (gitignored). Number them (-v1, -v2) instead of overwriting: the user compares versions by eye, and deleting them loses the history.

Verification that pays off

  • Prove the check can fail. Run the same harness against the previous build — BUNDLE=/tmp/old.js node tools/…, where the old bundle comes from git show HEAD:<artifact> — and put the before/after pair in the report or the commit message. A check that cannot fail proves nothing.
  • Compare lint against HEAD: git stash && npx eslint <file>; git stash pop. The repo has pre-existing errors (Css.js, WindowSettings.js), so the only question worth answering is whether your change adds to them.
  • Test assets must be real. A hand-written base64 PNG that Chromium cannot decode turned a whole verification into a no-op: build such files in the page (canvas.toBlob) instead.
  • Count requests with page.on('request'), not inside a route handler: a request that bypasses the route otherwise looks exactly like "no request happened".
  • Measure, do not infer. CSP, board HTML and engine flags decide the outcome, and a harness run answers in minutes what reading cannot.

Storage contract

CfgSaver.saveObj(domain, fn) hands the stored object straight to fn, and callers write into fields such as stats (loadedCfg.stats.reply++). Anything that seeds or rewrites DESU_Config must keep that shape complete: a partial object makes the reply submit throw before it does anything visible.

Form and layout work

  • The reply form is the board's own markup, wrapped: #de-pform holds it, .de-parea holds the open/close buttons, _makeHideableContainer()/setReply() decide where it sits. Board flags: qBottomAnchor (where the bottom area goes — a board may keep its own block at the end of the delform) and formHelpUrl (the board's page explaining its markup).
  • Any regrouping of fields must keep a per-field wrapper that PostForm.getFieldWrap() (closest(qFormTr + ', .de-altcell')) finds: hideField, Files.fileTr and Captcha.parentEl all work per wrapper. Move a control without its label (<label>, <th>) and the label stays behind; give two fields one shared wrapper and hiding one hides both.
  • Every file input needs its own wrapper (FileInput._wrap), and in thumbnail mode the strip lives in files.thumbsEl, not next to the input.
  • The captcha container is empty while the layout is built: Dollchan's Captcha empties it and restores it on focus, so its cell has to stay in the layout while empty — otherwise the captcha is restored into a detached node and never shows.
  • The reply form is as wide as its textarea: Cfg.textaWidth, or 70% of the viewport when that is 0 (the old default of 300px is what made a form spanning 70% of the page look narrow). The container (#de-pform) and the quick reply box only hug the content, which is why the board's own form is sized by its message field too. Never give the container a width and the form a percentage of it: the form's percentage then resolves against the container and the two disagree — a block wider than the form. Because the container ends up narrower than the area it sits in, it needs margin: 0 auto to stay centred. The alternative layout is the exception: its rebuilt form has no content of its own to be sized by, so there the container carries 70% of the page and the form fills it.
  • localStorage.deDebug = 1 makes PostForm.logState() print one dE form: JSON line per placement of the reply form — version and commit, script handler, the form's width and that of its table and textarea, every ancestor with its width, the alt-layout rows, the fields and the settings that shape them. Ask a user for that line instead of guessing at their state.
  • A form can be display: inline-block (the board's own), so a percentage width on something inside it resolves against nothing: widths belong on the form itself.

endchan (LynxChan) facts that cost the most time

  • A reply goes to /.api/replyThread, not to the form's action, with a JSON body ({"parameters":{…},"auth":{}}), and the answer is {"auth":null,"status":"ok","data":"<post number>"} — that data is where postNum comes from.
  • The thread page can be served without the reply that was just stored; hence the refresh loop that asks again (three times, a second apart) until that number appears.
  • Post numbers live in the deletion checkbox name (board-thread-post), not in an element id, and a post without .linkQuote makes the Post constructor throw.
  • img-src allows data: but not blob:; media-src falls back to default-src, which allows neither — video previews cannot be shown there at all.
  • The page's own footer (navigation, layout/colour selects, delete/report, board links) sits inside the same form[action$="contentActions.js"] that wraps the posts, which is why a reply area placed after the posts ends up nested in that form — harmless, since a control belongs to its nearest ancestor form.