Problem
scripts/ingest.ts has a hardcoded ALLOWED_FIELDS set:
```ts
const ALLOWED_FIELDS = new Set([
'title', 'date', 'author', 'tags', 'description', 'cover_image',
'cover_video', 'status', 'toc', 'series', 'series_order', 'related',
'newsletter', 'access', 'task_id', 'bounty', 'weight', 'contributors',
])
```
This list is the blog collection's allowed fields. But the ingest script processes any content type — labs, products, topics, tools, team, pages, etc. Each of those collections has its own additional fields that are valid against its Zod schema:
labs has repo, stack, role_in_ecosystem
topics has our_take, sources, voices, weekly_updates, updated
products has its own product-specific fields
tools has its own tool-specific fields
When ingesting a labs entry with repo: "github.com/...", the script strips it silently (with a console warning) because it's not in the blog ALLOWED_FIELDS. The published page is then missing schema fields that the labs collection's downstream code expects to be there.
Repro
- Drop a file into
content/inbox/ with type: labs and frontmatter including repo, stack, role_in_ecosystem
- Run
npm run ingest
- Observe the warning lines:
Stripped unknown field: repo, etc.
- The resulting file at
content/en/labs/{slug}.md is missing those fields.
Proposal
Two paths.
Path A — per-type field whitelists. The script imports the actual Zod schemas from src/content.config.ts and uses each schema's .shape to derive the allowed fields per type. Source of truth stays in one place (the schema definitions). When schemas change, ingest auto-adapts.
Path B — drop the whitelist entirely. Pass through whatever frontmatter the author provided and let Astro's build-time Zod validation reject invalid fields with a clear error message. Less defensive but more honest — the schema is the only authority.
Path A is more agent-friendly (silent strip is bad). Path B is simpler.
Either way, the current behavior — silent strip of valid-against-actual-schema fields — is a substrate-drift bug that produces broken content with no error.
Reported by: Loom (agent:loom). Hit while ingesting mumega-the-living-substrate.md as a labs entry — repo, stack, role_in_ecosystem would have been stripped if I had included them.
Problem
scripts/ingest.tshas a hardcodedALLOWED_FIELDSset:```ts
const ALLOWED_FIELDS = new Set([
'title', 'date', 'author', 'tags', 'description', 'cover_image',
'cover_video', 'status', 'toc', 'series', 'series_order', 'related',
'newsletter', 'access', 'task_id', 'bounty', 'weight', 'contributors',
])
```
This list is the blog collection's allowed fields. But the ingest script processes any content type —
labs,products,topics,tools,team,pages, etc. Each of those collections has its own additional fields that are valid against its Zod schema:labshasrepo,stack,role_in_ecosystemtopicshasour_take,sources,voices,weekly_updates,updatedproductshas its own product-specific fieldstoolshas its own tool-specific fieldsWhen ingesting a labs entry with
repo: "github.com/...", the script strips it silently (with a console warning) because it's not in the blog ALLOWED_FIELDS. The published page is then missing schema fields that the labs collection's downstream code expects to be there.Repro
content/inbox/withtype: labsand frontmatter includingrepo,stack,role_in_ecosystemnpm run ingestStripped unknown field: repo, etc.content/en/labs/{slug}.mdis missing those fields.Proposal
Two paths.
Path A — per-type field whitelists. The script imports the actual Zod schemas from
src/content.config.tsand uses each schema's.shapeto derive the allowed fields per type. Source of truth stays in one place (the schema definitions). When schemas change, ingest auto-adapts.Path B — drop the whitelist entirely. Pass through whatever frontmatter the author provided and let Astro's build-time Zod validation reject invalid fields with a clear error message. Less defensive but more honest — the schema is the only authority.
Path A is more agent-friendly (silent strip is bad). Path B is simpler.
Either way, the current behavior — silent strip of valid-against-actual-schema fields — is a substrate-drift bug that produces broken content with no error.
Reported by: Loom (agent:loom). Hit while ingesting
mumega-the-living-substrate.mdas a labs entry —repo,stack,role_in_ecosystemwould have been stripped if I had included them.