wintermeyer · wintermeyer · Sep 11, 2026 · Sep 11, 2026
@@ -62,8 +62,11 @@ jobs:
       # pdftoppm renders the qualification proof-document PDFs
       # (Vutuv.QualificationDocument); without it the PDF test paths
       # capability-skip and lose their coverage.
-      - name: Install poppler-utils (PDF rendering) and ffmpeg (video on posts)
-        run: sudo apt-get update && sudo apt-get install -y --no-install-recommends poppler-utils ffmpeg
+      # qpdf joins them for issue #2107: it is what takes the author's name,
+      # the software and the dates out of a PDF, and several fixtures already
+      # built their hostile PDFs with it and skipped themselves without it.
+      - name: Install poppler-utils (PDF rendering), ffmpeg (video on posts) and qpdf (PDF metadata)
+        run: sudo apt-get update && sudo apt-get install -y --no-install-recommends poppler-utils ffmpeg qpdf
 
       - name: Set up Elixir/OTP
         uses: erlef/setup-beam@v1

@@ -897,7 +897,12 @@ config :vutuv, :attachments,
   render_concurrency: 1,
   pdfinfo: "pdfinfo",
   pdfdetach: "pdfdetach",
-  pdftoppm: "pdftoppm"
+  pdftoppm: "pdftoppm",
+  # `qpdf` takes the author's name, the software and the dates out of a PDF
+  # (issue #2107). Optional like the rest: without it the composer hides that
+  # switch rather than offering an answer this installation cannot honour, and
+  # every file is served exactly as it was uploaded.
+  qpdf: "qpdf"
 
 # Job postings (Vutuv.Jobs, milestone 11).
 #   * default_runtime_days — how long a published posting stays live before it

@@ -968,7 +968,8 @@ if config_env() == :prod do
                attachment_defaults[:render_concurrency],
            pdfinfo: System.get_env("PDFINFO_PATH") || attachment_defaults[:pdfinfo],
            pdfdetach: System.get_env("PDFDETACH_PATH") || attachment_defaults[:pdfdetach],
-           pdftoppm: System.get_env("PDFTOPPM_PATH") || attachment_defaults[:pdftoppm]
+           pdftoppm: System.get_env("PDFTOPPM_PATH") || attachment_defaults[:pdftoppm],
+           qpdf: System.get_env("QPDF_PATH") || attachment_defaults[:qpdf]
          )
 
   # Post images are auth-proxied: the app checks the post's audience, then

@@ -48,6 +48,16 @@ Related documents: [README](../README.md) (overview) ·
   (`ATTACHMENT_PREVIEW_PAGES`); without it a PDF is still accepted and simply
   shows no preview.
   A check that cannot run is never treated as a check that passed.
+- **qpdf** (optional, `apt-get install qpdf`) — takes the author's name, the
+  software and the dates out of a PDF attached to a post, which is what the
+  composer's "Remove the metadata from the files" switch does. Without `qpdf`
+  on `$PATH` (or `QPDF_PATH`) that switch is not shown at all and every file is
+  served exactly as it was uploaded; nothing else changes. Debian stable ships
+  12.2.0, which is what the flags used here are chosen against. The private
+  copy of the upload is never rewritten, so a member who turns the switch off
+  gets their file back byte for byte. Note the modification date survives:
+  removing it as well would mean rebuilding the document from its pages alone,
+  which drops the bookmarks and the page labels of a long PDF.
 - **ffmpeg** (optional, `apt-get install ffmpeg`) — video on posts: converts
   a member's clip into the files browsers play and pulls the stills the AI
   check looks at. Debian's build carries `libx264` and `libsvtav1`; without
@@ -140,6 +150,7 @@ Everything else has a default (the vutuv.de production value):
 | `ATTACHMENT_MONTHLY_MB` | `500` | The same over any 30 days |
 | `ATTACHMENT_PREVIEW_PAGES` | `3` | How many of a file's first pages are shown as pictures under the post. A picture sent in a message always has exactly one preview page, which is the picture itself. `0` turns previews off; more than `5` is treated as `5`. A PDF page is rendered by `pdftoppm`, a text or Markdown file by the headless Chromium the link previews use — where neither is installed the file simply shows no preview, and nothing else changes |
 | `ATTACHMENT_RENDER_CONCURRENCY` | `1` | How many files have their preview pages rendered at once. Everything past that queues. Raise it on a machine with cores to spare |
+| `QPDF_PATH` | `qpdf` | The binary that removes a PDF's metadata, if not on `$PATH` under that name. Missing it means the composer does not offer the switch and files are served exactly as uploaded |
 | `PDFINFO_PATH` / `PDFDETACH_PATH` / `PDFTOPPM_PATH` | `pdfinfo` / `pdfdetach` / `pdftoppm` | The three poppler binaries — the first two run the PDF check, the third renders preview pages — if not on `$PATH` under those names. Missing either of the first two means PDFs are not offered (see the dependency list above); missing the third only means no preview pages |
 | `SCREENSHOT_BLOCKLIST` | – | Extra pages never to take a link-preview screenshot of, on top of the shipped `reddit.com` and `heise.de`. Comma-separated domains and/or URLs, copied into the blocklist table the first time you migrate; afterwards the live list is edited in the admin area (see "Screenshot blocklist" below) and this variable is inert. `SCREENSHOT_BLOCKED_HOSTS` is the older name and still works |
 | `SCREENSHOT_PAGE_CHECK` | `true` | Whether each link-preview capture is judged by the Ollama vision model on whether it shows the page or a consent / ad / login wall or a bot check, and the site blocklisted when it does not (see "The list mostly writes itself" below). `false` leaves the blocklist entirely hand-written — the setting for an installation without Ollama. Independent of `IMAGE_MODERATION_ENABLED`: that one is the safety gate, this one is a quality filter |
@@ -884,7 +895,17 @@ every member who has not decided for themselves is unnamed, with the switch
 still theirs to turn back on. It never changes the **count**: a post always
 shows how many likes it got, whoever may be named.
 
-A third is about how loud the feed is: **whether the source tabs quote what
+A third is the one an intranet is most likely to want the other way round:
+**whether search engines and AI may read what a member posts**. vutuv ships this
+on, because the site is a public network and a post is meant to be found. Turn
+*Search engines and AI may read my posts* off at `/admin/preferences` and every
+member who has not decided for themselves publishes posts that ask crawlers to
+stay away — and that are **not sent to other networks at all**, because a server
+elsewhere keeps its copy for good and no instruction of ours reaches it. It
+applies to posts written from then on: the answer is stored on each post as it
+is published, so nothing already written changes, in either direction.
+
+A fourth is about how loud the feed is: **whether the source tabs quote what
 lands on the one a member is not reading**. Something arriving on the tab they
 are not on marks it with a dot either way; on top of that, the bar can quote
 the arrival — author and first words — for a few seconds before folding back.
@@ -893,7 +914,7 @@ feed turns *Quote what arrives on the other tab* off at `/admin/preferences`,
 or shortens the window there; the dot is unaffected, and every member can set
 both for themselves.
 
-A fourth is about what the site costs to load: **data-saving mode**
+A fifth is about what the site costs to load: **data-saving mode**
 (`low_bandwidth?`, its own page at `/settings/bandwidth`). A member with the
 mode on gets three things. Every post photo, picture from another network,
 URL screenshot and profile cover loads as its **lite version** — the same

@@ -43,6 +43,19 @@ agents/LLMs → `ai-train=`/`ai-input=`, robots `noai, noimageai`) — any
 combination is valid; pages that are noindexed page-level (profile sections,
 people lists, restricted posts) send every signal as `no`.
 
+A **post** carries a third choice of its own, `posts.noindex_noai?` (issue
+#2107): one switch for both machine audiences, taken once by the member on
+`/settings/privacy` and stamped onto the row at publish time. It only ever
+*adds* — `PostDoc.robots_axes/3` ors it into the author's two, and nothing about
+a post re-opens what its author closed. A post that says no also leaves the
+sitemap, both RSS feeds, every shared listing document and the Fediverse
+entirely; where it still appears inside somebody else's document (a
+conversation, a reply, a repost, a pinned-post excerpt) its words are replaced
+by one sentence rather than the row being dropped, so the totals stay honest.
+The HTML conversation on a permalink is the deliberate exception and shows it,
+because the promise is about machines and not about readers. See
+[attachments.md](attachments.md#machines-and-the-metadata-in-a-file-2107).
+
 The per-user detail sub-pages (`/:slug/emails`, `/tags`, `/work_experiences`,
 `/followers`, …) are kept out of search by that page-level `X-Robots-Tag:
 noindex` (`VutuvWeb.Plug.NoIndex` on the `:user_pipe` pipeline), **not** by a

@@ -60,10 +60,11 @@ is that page 1 renders. Neither calls it yet.
 `Vutuv.AttachmentStore` keeps the upload verbatim in the private
 `originals/attachments/<token>/` tree and a served copy under
 `attachments/<token>/`, both keyed by the row's URL token, never its id.
-They are byte-identical today; they are still two files, because the served
-copy is what #2107 rewrites when an author asks for the metadata to be
-removed, and the private tree's promise — this is exactly what the member sent
-— has to survive that. Neither tree gets a `Plug.Static` mount: every served
+The served copy is a **derivation** of the original: it is what
+`Vutuv.Attachments.Metadata` rewrites when a PDF has its metadata taken out
+(#2107), while the private tree keeps its promise through all of it — that is
+exactly what the member sent, byte for byte. Neither tree gets a `Plug.Static`
+mount: every served
 byte will go through an authorizing proxy (#2108). Both are gitignored, and
 `test/vutuv/uploads_gitignore_test.exs` fails the build if that slips.
 
@@ -93,7 +94,7 @@ Everything is per installation, read in `config/runtime.exs` with the
 `ATTACHMENT_MAX_MB`, `ATTACHMENTS_PER_POST`, `ATTACHMENT_DAILY_MB`,
 `ATTACHMENT_MONTHLY_MB`, `ATTACHMENT_PREVIEW_PAGES`,
 `ATTACHMENT_RENDER_CONCURRENCY`, `PDFINFO_PATH`, `PDFDETACH_PATH`,
-`PDFTOPPM_PATH`.
+`PDFTOPPM_PATH`, `QPDF_PATH`.
 
 `ATTACHMENT_UPLOADERS` is `admins` while the milestone is being built, the way
 video was introduced: a post cannot show or hand out its files until #2108
@@ -117,6 +118,116 @@ what keeps the daily sweep and a re-mounted composer off it. Setting `post_id`
 clears it in the same statement, so exactly one column ever answers "who holds
 this file".
 
+## Machines, and the metadata in a file (#2107)
+
+### One standing decision, stamped onto each post
+
+**"Search engines and AI may read my posts"** is a `Vutuv.Prefs` key,
+`posts_machines_allowed?`, on `/settings/privacy`. It is asked **once**, not per
+post: a composer that asked would put the same question in front of a member
+several times a day, and the answer almost never changes between two posts.
+
+The member's answer is copied onto the row **as the post is published**
+(`Vutuv.Posts` private `seed_post/1`, the seed struct both member-post paths
+build from) into `posts.noindex_noai?`, read back through
+`Vutuv.Posts.machines_allowed?/1` and its query twin
+`scope_machines_allowed/1`. Nothing re-reads the setting when a post is
+rendered, and that is the design rather than an optimisation: a live read would
+mean flipping the switch silently rewrote every older post, and for the ones
+already federated it would promise a withdrawal nothing can perform. So each
+post keeps what it went out with, an edit keeps it too, and the setting is only
+ever the default for the next one. An explicit `noindex_noai` attr still wins,
+which is what `POST /api/2.0/posts` sends.
+
+A **page's** post does not take it: an organization has its own `seo?`/`geo?`,
+and one publisher's private posture must not mute the brand they publish for.
+
+A pref rather than a plain column because `nil` has to mean "never asked":
+`noai?`'s column default is `true`, so 5,867 of 6,026 members on the dev copy of
+production "carry" an answer nobody gave, and nothing can tell that from a real
+one. Here NULL inherits the installation default, which an operator moves at
+`/admin/preferences` — worth having, since an intranet installation may well
+want the opposite posture.
+
+### What "no machines" costs, and why the settings page says it
+
+Saying no means `noindex, noai, noimageai` on the permalink and on every
+`.md`/`.txt`/`.json`/`.xml` sibling, no sitemap entry, no place in the two RSS
+feeds or on a tag page (both carry one all-yes `Content-Signal` for a whole
+list), and no first line quoted on any shared listing surface. And it means the
+post **is not federated at all** — not the Create, not a later Update, not the
+unfreeze's republish, not a topic actor's Announce, not somebody else's boost,
+not the `featured` collection, not the outbox count, and not the Note served on
+request to an ActivityPub `Accept` header. A header is advisory and re-read on
+every visit; a copy on somebody else's server is neither, so this has to be a
+gate and not a directive. That sentence is the registry `hint` on
+`posts_machines_allowed?`, so the member and the admin read the same words.
+
+### The chokepoint
+
+**A post's body leaves the database toward a surface a machine can read only
+through a query that has been piped through `Vutuv.Posts.scope_machines_for/2`**
+(or its anonymous case `scope_machines_allowed/1`). That is the invariant, and
+it is enforced rather than remembered: `test/vutuv/post_body_chokepoint_test.exs`
+reads the source, collects every function under `lib/vutuv/` that selects a post
+`body`, and fails the build for any that does not pipe through the gate.
+
+It is a property and not a list because three review rounds each produced a
+complete measurement and each missed the next surface — the archive, the tag
+pages and the calendar; then the conversation, the reply, the repost and the
+profile document; then the profile HTML. The last pair is why the gate is in the
+**query**: `recent_posts_by_authors/3` selects `body:` into bare maps, which no
+struct-taking helper can reach, and `pinned_post/2` fetches by id rather than
+through a timeline, so no listing gate applied. Both put a withheld post's whole
+text on `/:slug`, which is public, anonymous, in the sitemap and carries no
+`X-Robots-Tag`.
+
+`scope_visible/2` is its mandatory partner and answers a different question — may
+this **viewer** see it — so neither substitutes for the other. An exception is
+explicit and carries its reason in the test's own `@deliberate` and
+`@exempt_queries` maps; the silent absence of a check is what the guard exists to
+stop.
+
+### Where a withheld post is redacted, and where it is not
+
+The switch protects the post's own documents; what it did not protect at first
+was the post as it appears **inside somebody else's**, because a document's
+robots axes are computed from its *subject* post and a conversation quotes other
+people's. So `PostDoc` redacts **per entry**: `thread_entries/1` and
+`reply_entry/1` hand back the sentence "Post not open to search engines" in
+place of a withheld body, `withheld_excerpt/1` does the same for a quoted line
+(`ProfileDoc` shares it for the pinned post), the reposts leg of
+`author_timeline_query/3` drops a withheld post outright, and the archive's
+one-level ancestor fallback stops printing a withheld parent above a reply card.
+
+The **HTML conversation on a permalink still shows every word**. Page and doc
+differ there on purpose: the promise is about machines, not about readers. The
+archive listing is the one place they must agree, because that page is a crawl
+surface — which is also why its withheld rows leave it.
+
+### The metadata in a file
+
+**"Remove the metadata from the files"** is `posts.strip_metadata?`, the
+author's *answer*, and the one question the composer still asks — it is about
+*these* files, so it belongs beside them, and it only shows on a new post
+carrying one. What actually happened to a given file is
+`attachments.metadata_stripped_at`. `Vutuv.Attachments.Metadata` is the one
+place vutuv talks to `qpdf`, probed once per VM the way ffmpeg is, and the
+switch is hidden where the binary is absent **or too old** — the probe asks
+qpdf whether it knows the flags (`--help=--remove-info`), not whether the file
+exists, because Debian 12 ships 11.3.0 and Ubuntu 24.04 ships 11.9.0 while
+`--remove-info` arrived in 11.10.0. The run happens at **intake**, on every PDF,
+deriving the served copy from the verbatim original with
+`--remove-info --remove-metadata --deterministic-id --linearize`; an author who
+says no gets the original copied back at claim time, which costs a `File.cp`
+rather than a shell-out on the publish path. `--remove-structure` is
+deliberately not used (that is the tagged-PDF tree a screen reader follows),
+and `/ModDate` survives by qpdf's own design.
+
+That answer travels through `post_drafts.strip_metadata?`, so a reload brings it
+back beside the words it was chosen with; it does not count as content, so
+flipping it on an empty composer creates no draft.
+
 ## The post waits for its files
 
 A post carrying a file is not published until the server is done with every one

@@ -151,13 +151,27 @@ confirmation flow are described in
 ### Member preferences with installation defaults (`Vutuv.Prefs`)
 
 The map and post-display settings are the first citizens of the generic
-**preferences system**; the third group is `:privacy`, which holds
+**preferences system**; the third group is `:privacy`, which holds two.
 `like_attribution?` — whether a post permalink names this member among the
 people who liked it (issue #1233, see the "Who liked it" section in
-[posts-and-feed.md](posts-and-feed.md)). It is the one pref whose home is not
-`/settings/preferences` but the visibility page (`/settings/privacy`, reset via
-POST `/settings/privacy/reset`), because it is a privacy posture rather than a
-display detail. The `:bandwidth` group holds one knob with a different shape
+[posts-and-feed.md](posts-and-feed.md)). And `posts_machines_allowed?` — whether
+search engines and AI may read the posts this member writes from now on (issue
+#2107): the answer is copied onto each post as it is published, so changing it
+leaves everything already written exactly as it went out, and saying no also
+keeps a new post off the other networks entirely, which the card's hint says in
+words because nothing else can tell the member. The detail is in
+[attachments.md](attachments.md). Neither pref's home is
+`/settings/preferences`: both live on the visibility page (`/settings/privacy`,
+reset via POST `/settings/privacy/reset`), because they are privacy postures
+rather than display details.
+
+Note the shape difference on that page. The two crawler switches above them,
+`noindex?` and `noai?`, are plain opt-**out** columns rendered positively with
+`checked_value: "false"`, while a `Vutuv.Prefs` boolean is stored the way it
+reads — which is why the posts setting is named positively and sits in a card of
+its own rather than as a third row in theirs.
+
+The `:bandwidth` group holds one knob with a different shape
 again: `low_bandwidth?` (data-saving mode) decides whether this member's
 browser is ever told where the 155 kB WYSIWYG editor bundle lives (see
 "Low-bandwidth mode" in [posts-and-feed.md](posts-and-feed.md)) and whether