feat(postgrest-typegen): add package with multi-language code generation#302
feat(postgrest-typegen): add package with multi-language code generation#302avallete wants to merge 16 commits into
Conversation
Add the @supabase/postgrest-typegen package: the PostgREST type-generation engine extracted from postgres-meta into a driver-agnostic library. This scaffold establishes the contract and tooling; the introspection and language generators land in follow-up work (PGMETA-106/107/108/110) and currently throw "not implemented yet". - Hard split between introspection (db -> GeneratorMetadata) and generation (GeneratorMetadata -> source string), with subpath exports `./introspection` and `./generation`. - `src/types.ts` ports postgres-meta's Postgres* metadata interfaces and the pluggable `GeneratorMetadata` contract. - Public signatures: `introspect`, `generateTypescript`, `generateGo`, `generatePython`, `generateSwift`. - Tooling wired up: tsc build (dist + d.ts), bun:test runner, oxfmt/oxlint, knip. prettier pinned exact to 3.5.3 for byte-parity with postgres-meta. - Smoke test pins the public API surface. No changeset: all implementations are throwing stubs, so the package must not publish yet. Functional changesets accompany the implementation tickets. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
Port postgres-meta's Go and Python type-generation templates into
src/generation/{go,python}.ts as pure generateGo()/generatePython()
functions over the GeneratorMetadata contract. Behavior-preserving:
output is byte-identical to postgres-meta's templates for the same
inputs (verified by cross-running both implementations over shared
fixtures covering enums, arrays, composite types, nullability/identity/
generated columns, and views/matviews/foreign tables).
Add a shared fixture builder (test/generation/fixtures.ts) ported from
postgres-meta's go.test.ts pattern, plus per-language inline-snapshot
unit tests (no Docker). Update the API-surface smoke test now that the
Go and Python generators are implemented (TypeScript/Swift remain
stubs for PGMETA-107).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
Port postgres-meta's TypeScript and Swift type-generation templates into
src/generation/{typescript,swift}.ts:
- generateTypescript(metadata, opts?) — async (prettier 3.5.3, pinned exact).
Options: detectOneToOneRelationships, postgrestVersion (→
__InternalSupabase.PostgrestVersion), and defaultSchema (default 'public',
replacing postgres-meta's GENERATE_TYPES_DEFAULT_SCHEMA env read). Also
exports pgTypeToTsType.
- generateSwift(metadata, opts?) — accessControl 'internal'|'public'|
'private'|'package' (default 'internal'). Dropped the dead prettier import.
- Restore src/generation/constants.ts (VALID_FUNCTION_ARGS_MODE,
VALID_UNNAMED_FUNCTION_ARG_TYPES) now that the TS generator consumes it.
Behavior-preserving: output is byte-identical to postgres-meta's templates,
verified by cross-running both implementations over shared fixtures across
default / detectOneToOneRelationships / postgrestVersion (TS) and internal /
public access control (Swift), covering tables, enums, arrays, composite
types, views, matviews, foreign tables, relationships, and multi-schema.
Add per-language inline-snapshot unit tests (relationships + is_one_to_one,
detect flag on/off, postgrestVersion, function signatures, defaultSchema,
Swift accessControl) plus shared relationship/function fixtures. Update the
API-surface smoke test now that all four generators are implemented.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
…malize Port the typegen-relevant introspection SQL builders from postgres-meta into src/introspection/sql/ (schemas, tables, foreign-tables, views, materialized-views, columns, functions, types, table-relationships, views-key-dependencies), plus helpers.ts (filterByList, DEFAULT_SYSTEM_SCHEMAS) and common.ts (the prop shapes used by the generator path). Builders are ported verbatim and parameterized; introspect() (PGMETA-110) calls them with the single generator-path option combination (system schemas excluded for schemas/tables/views/ columns/functions/relationships; no default exclusion for foreign tables and materialized views; types with includeTableTypes + includeArrayTypes and no schema filter — the unfiltered query is intentionally not optimized, templates need system/array types). Add src/introspection/normalize.ts replicating postgres-meta's global int8 type parser (src/lib/db.ts): known int8 top-level fields (id, table_id, type_relation_id, return_type_id, return_type_relation_id, bytes, live_rows_estimate, dead_rows_estimate) are coerced string→number when safe-integer, else kept as string — so GeneratorMetadata is identical under any driver. int8 values inside jsonb columns are already numbers via JSON decoding and need no coercion; composite text ids (a column's "<oid>.<attnum>") are not safe integers and pass through untouched, matching the parser exactly. Verified the 10 ported builders produce byte-identical SQL to postgres-meta for the generator-path option combination. Add SQL-builder inline-snapshot tests (pinning that combination) and normalize unit tests asserting equivalence to db.ts's parser. Drop pg-format/@types/pg-format from knip ignoreDependencies now that helpers.ts imports pg-format. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
Port postgres-meta's PostgresMetaRelationships.list() into src/introspection/relationships.ts. This is the PostgREST cartesian-product view-relationship logic — pure TypeScript, not SQL — that maps a base table's foreign-key columns onto the view columns that select them, producing view→table, table→view, and view→view relationships. The cartesian-product expansion is extracted into a pure expandViewRelationships(tableRelationships, viewsKeyDependencies) function (ported verbatim) so it can be unit-tested in isolation — something untested upstream. listRelationships(db, opts) runs TABLE_RELATIONSHIPS_SQL + VIEWS_KEY_DEPENDENCIES_SQL through the injected Queryable (errors throw; callers adapt) and concatenates table relationships with the view-derived ones, mirroring upstream with the generator path's includeSystemSchemas: false default. Verified byte-identical output against postgres-meta's PostgresMetaRelationships.list() for a fixture exercising all three relationship kinds plus a 2x2 composite-key cartesian product. Unit tests cover view→table, table→view, view→view, the multi-column cartesian product, the empty case, and listRelationships wiring. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
Implement src/introspection/index.ts — introspect(db, opts) — a port of getGeneratorMetadata from postgres-meta. It calls the ported SQL builders directly (no manager classes) with the single generator-path option combination, applies the int8 normalize coercion to every result set, runs the relationships logic, and assembles GeneratorMetadata. It replicates getGeneratorMetadata's post-filtering (schema include/exclude, dropping trigger/event_trigger functions) but leaves connection lifecycle to the caller — the injected Queryable's errors just throw. pg.Pool satisfies the structural Queryable directly and returns int8 as strings, so the integration test exercises normalize.ts end-to-end. Integration tests (testcontainers + real Postgres) apply the copied 00-init.sql / 01-memes.sql fixtures and assert: - public schema present, system schemas excluded - int8 ids normalized to numbers; a column's composite "<oid>.<attnum>" id stays a string - users columns introspect with expected formats/flags (inline snapshot) - trigger/event_trigger functions excluded; plain SQL `add` kept - enum + composite types present - all three view-relationship kinds (view→table, table→view, view→view) - includedSchemas / excludedSchemas filter behavior The container is configured with the `postgres` superuser to match the fixtures (testcontainers defaults to `test`), and uses Wait.forHealthCheck() per the repo convention. Container lifecycle lives in the test's beforeAll/afterAll (kept lazy) so the pure unit tests still run without Docker. Clear knip ignoreDependencies now that pg/ testcontainers are imported by the integration test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
Add test/parity/: introspect the shared postgres-meta fixture DB
(00-init.sql / 01-memes.sql) on postgres:15-alpine, run all four
generators with default options, and assert byte-equality against
committed golden files in expected/.
The golden files were captured verbatim from the current postgres-meta
CLI run against the same fixtures:
docker run ... postgres:15-alpine # empty container
<apply fixtures via pg Pool.query, as the test does>
PG_META_DB_URL=postgres://postgres:postgres@localhost:55432/postgres \
PG_META_GENERATE_TYPES={typescript,go,python,swift} \
node --loader ts-node/esm src/server/server.ts > expected/<lang>.txt
All four were verified byte-identical to this package's introspect →
generate output against the same DB. postgres-meta's CLI prints via
console.log, which appends exactly one trailing newline to the
generator's return value, hence the `+ "\n"` in the assertions.
Note on ordering: postgres-meta's go/python/swift templates emit views in
the order the (unsorted) views query returns them, i.e. DB physical
order. That order depends on how the fixtures are loaded, so the golden
was captured from a DB loaded via Pool.query exactly as the test loads
it; this is deterministic across fresh postgres:15-alpine instances
(verified by repeated runs). The golden files are stored as .txt so
oxfmt/oxlint/tsc do not reformat them and break byte-parity.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
…ty loop Add scripts/verdaccio-publish-postgrest-typegen.ts (+ the `postgrest-typegen:publish-local` root script), mirroring the pg-delta publish-local helper. It builds the package, publishes a fresh `0.0.0-local.<unix-ts>` version to the locally-running Verdaccio, and restores the working-copy version in a `finally` so the repo stays clean. This enables the Phase 2 byte-parity loop (PGMETA-112): publish locally, point postgres-meta at the local registry via a scoped (uncommitted) .npmrc, and run postgres-meta's typegen snapshot suite before anything is published to npm. Verified end-to-end: `bun run verdaccio:start` + publish-local produces an installable tarball, and postgres-meta installs it from Verdaccio and imports the compiled dist (introspect + generateTypescript/Go/Python/Swift + pgTypeToTsType all resolve via the package's `import` export condition). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
- detect-changes action: add `postgrest-typegen` / `postgrest-typegen-package-json` filters + outputs, and exclude the package's package.json/CHANGELOG from the release-PR content gate. - tests.yml: thread the new detect-changes outputs; add gated `check-types`/`knip` steps for postgrest-typegen; add a `postgrest-typegen-tests` job (cloned from pg-topo-tests, Docker-backed via testcontainers); wire it into the coverage and release-preview aggregators; add ./packages/postgrest-typegen to the pkg-pr-new publish list. - Add a `minor` changeset so changesets pre-release mode publishes the alpha on merge. - Update the canonical agent doc (.github/agents/pg-toolbelt.md; AGENTS.md and CLAUDE.md are symlinks) Packages + CI sections. Final quality pass green: bun run format-and-lint, check-types, knip. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
…h opt-in validator
Back the GeneratorMetadata contract with ArkType as the single source of
truth: each Postgres* shape in src/types.ts is now an ArkType schema and the
exported type is `typeof schema.infer`. The GeneratorMetadata array fields use
`.omit("columns")` to mirror the previous `Omit<…, "columns">`.
Add an opt-in runtime validator so integrators producing GeneratorMetadata via
a custom/injected introspection adapter can verify the result instead of
blindly casting:
- `parseGeneratorMetadata(data: unknown): GeneratorMetadata` — throws a
TypeError with ArkType's readable summary on mismatch.
- `generatorMetadataSchema` — exported for custom flows.
Validation is intentionally NOT baked into `introspect()` (the result is still
returned casted); callers wrap it when they want the guarantee.
The inferred types are structurally identical to the previous hand-written
interfaces — pinned by a compile-time equivalence test
(test/validation/validate.test.ts) that fails to compile on any drift — so the
generators and postgres-meta's consumed shapes are unchanged and generator
output stays byte-identical (the parity test still passes). Adds `arktype`
2.2.1 as a dependency.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
…ary_keys The per-table `relationships` (PostgresRelationshipOld) and `primary_keys` fields on `PostgresTable`/`GeneratorMetadata.tables[]` were never read by any generator: TypeScript builds relationships from the top-level `GeneratorMetadata.relationships`, and Go/Python/Swift emit none. Remove the fields, the `PostgresRelationshipOld`/`PostgresPrimaryKey` types, and the matching `pg_constraint` JSON-array join + primary-key subquery from the tables introspection SQL. Generator output is unchanged (parity test green); the tables query is now cheaper. postgres-meta's `/tables` REST endpoint is unaffected (it uses its own table types, not this package's). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
🦋 Changeset detectedLatest commit: 11ff444 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
avallete
changed the title
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 53b61784ed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| unnest(arg_modes) as mode, | ||
| unnest(arg_names) as name, | ||
| unnest(arg_types)::int8 as type_id, | ||
| unnest(arg_has_defaults) as has_default |
There was a problem hiding this comment.
When introspecting functions with OUT/TABLE args, such as RETURNS TABLE (...), arg_modes/arg_types include those output arguments but arg_has_defaults is only sized from pronargs (input arguments). This unnest therefore pads has_default with NULL for the output rows, while PostgresFunction.args[].has_default is validated as a boolean, so parseGeneratorMetadata(await introspect(...)) rejects legitimate schemas containing returns-table functions. Coalesce the padded value to false or relax the validator to match the introspector output.
Useful? React with 👍 / 👎.
| let output = ` | ||
| package database | ||
|
|
||
| ${tables |
There was a problem hiding this comment.
When metadata contains foreignTables, the Go generator only iterates regular tables before views/materialized views, so FDW relations never get Select/Insert/Update structs. PostGREST can expose foreign tables and the TypeScript/Swift generators treat them as table-like objects, so Go users get incomplete schema types whenever their exposed schema contains a foreign table.
Useful? React with 👍 / 👎.
| const ctx = new PythonContext(types, columns, schemas); | ||
| // Used for efficient lookup of types by schema name | ||
| const schemasNames = new Set(schemas.map((schema) => schema.name)); | ||
| const py_tables = tables.flatMap((table) => { |
There was a problem hiding this comment.
When metadata contains foreignTables, Python generation maps only tables and then views/materialized views, so exposed FDW relations are silently omitted and no Pydantic/TypedDict classes are produced for them. This leaves Python output incomplete for schemas using foreign tables, unlike the TypeScript/Swift generators that include foreignTables as table-like entries.
Useful? React with 👍 / 👎.
| const enumType = types.find( | ||
| (type) => type.name === pgType && type.enums.length > 0, | ||
| ); |
There was a problem hiding this comment.
For an included table whose column uses an enum/composite type from a schema that is not included, introspect() still returns all types, but generateSwift only emits enums/composites for each included schema. This name-only lookup can therefore find the excluded type and line 403 emits a bare Status/AddressSelect reference inside the included schema without emitting that declaration there, causing the generated Swift to fail to compile for valid cross-schema type usage.
Useful? React with 👍 / 👎.
| return relationships.filter( | ||
| (relationship) => | ||
| relationship.schema === object.schema && | ||
| relationship.referenced_schema === object.schema && |
There was a problem hiding this comment.
When callers include multiple exposed schemas and a table in one schema has an FK to a table in another, introspection returns that relationship with a different referenced_schema, but this predicate drops it before emitting Relationships. The generated TypeScript therefore omits valid PostgREST embeds for cross-schema foreign keys even though both schemas are present in Database.
Useful? React with 👍 / 👎.
| this.variants = type.enums; | ||
| } | ||
| serialize(): string { | ||
| const variants = this.variants.map((item) => `"${item}"`).join(", "); |
There was a problem hiding this comment.
PostgreSQL enum labels can contain characters such as " or \, but this interpolation writes them directly inside a Python string literal. A label like needs"review produces invalid generated Python in the Literal[...] alias, so enum labels should be string-escaped before being emitted.
Useful? React with 👍 / 👎.
| )} {`, | ||
| ...enum_.cases.map( | ||
| (case_) => | ||
| `${ident(level + 1)}case ${case_.formattedName} = "${case_.rawValue}"`, |
There was a problem hiding this comment.
PostgreSQL enum labels are emitted directly into Swift string literals here, so a valid label containing " or \ makes the generated enum fail to compile or changes the raw value. Escape the raw value before interpolation so arbitrary enum labels remain valid Swift source.
Useful? React with 👍 / 👎.
| } | ||
| Insert: { | ||
| ${columnsByTableId[table.id].map((column) => { | ||
| if (column.identity_generation === "ALWAYS") { |
There was a problem hiding this comment.
When a column is GENERATED ALWAYS AS, this branch only blocks ALWAYS identity columns, so the generated TypeScript Insert type still accepts values for stored generated columns. PostgreSQL rejects user-supplied values for generated columns, so those properties should be treated like ?: never rather than valid insert fields.
Useful? React with 👍 / 👎.
| not_required || | ||
| col.is_nullable || | ||
| col.is_identity || | ||
| col.default_value !== null, |
There was a problem hiding this comment.
For table Insert TypedDicts, optionality ignores col.is_generated, so generated columns remain accepted as normal insert fields (and can even be required with custom metadata that lacks a default expression). PostgreSQL rejects explicit values for generated columns, so the Python insert shape should mark them as omitted/Never instead of accepting the column type.
Useful? React with 👍 / 👎.
| type DefaultSchema = DatabaseWithoutInternals[Extract<keyof Database, ${JSON.stringify( | ||
| defaultSchema, | ||
| )}>] |
There was a problem hiding this comment.
If callers generate TypeScript for only non-public schemas without passing defaultSchema, or for empty metadata as the smoke test does, this resolves DefaultSchema through Extract<keyof Database, "public"> to never; the subsequent helper types index DefaultSchema["Tables"]/["Views"], producing invalid TypeScript. Fall back to an existing schema or emit guarded helpers when the configured default schema is not present.
Useful? React with 👍 / 👎.
…n args
introspect() emits has_default: null for the OUT columns of RETURNS TABLE
/ OUT-arg functions — the introspection SQL sizes arg_has_defaults from the
input-arg count (pronargs) while arg_modes/arg_types include output args, so
the trailing rows are padded with NULL. The opt-in parseGeneratorMetadata
validator typed has_default as a plain boolean and rejected that valid
introspector output. Widen the field to `boolean | null`.
Generator output is unaffected: the generators already treat has_default
truthily (`has_default ? "?" : ""`), so null and false behave identically.
Parity tests remain green.
RED (before fix):
validate.test.ts:
Invalid GeneratorMetadata: functions[0].args[1].has_default must be
boolean (was null)
introspect.test.ts (real fixtures):
functions[9].args[0].has_default must be boolean (was null)
functions[15].args[0].has_default must be boolean (was null)
functions[16].args[1].has_default must be boolean (was null)
Tests added (RED → GREEN):
- validate.test.ts: parseGeneratorMetadata accepts a table-mode arg with
has_default: null.
- introspect.test.ts: asserts function_returning_table's OUT columns carry
has_default === null, and parseGeneratorMetadata(full) does not throw.
Addresses Codex review comment on functions.sql.ts (the only one touching new
code rather than verbatim-ported behavior).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
Re: Codex automated review (13 comments)Thanks for the thorough pass. Triage below. TL;DR: 12 of the 13 describe behavior ported verbatim from postgres-meta (the Why most are intentionally deferredThis package is a byte-for-byte extraction of postgres-meta's type-generation engine. The stated constraint (README / package So each item below reproduces identically against today's
These are legitimate observations — they're just out of scope for a parity-preserving extraction. The one fixed here
I won't 👍/👎 each inline thread to avoid noise — happy to open tracking issues for the deferred items if that's preferred. Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 74e498fa39
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ([formattedName, type, name]) => { | ||
| return ` ${formattedName.padEnd(maxFormattedNameLength)} ${type.padEnd( | ||
| maxTypeLength, | ||
| )} \`json:"${name}"\``; |
There was a problem hiding this comment.
The Insert/Update structs make default/identity/nullable columns pointers (and all Update columns pointers), but this tag lacks omitempty, so encoding/json marshals every unset pointer as null. In normal PATCH usage like PublicTicketsUpdate{Name: &name}, all other fields are sent as null, which can clear data or violate NOT NULL; inserts also send null instead of letting defaults/identity values fire. Track optionality and emit json:"...,omitempty" or another omission mechanism for write structs.
Useful? React with 👍 / 👎.
| t.typrelid = 0 | ||
| or ( | ||
| select | ||
| c.relkind ${props.includeTableTypes ? `in ('c', 'r', 'v', 'm', 'p')` : `= 'c'`} |
There was a problem hiding this comment.
When includeTableTypes is true, this list includes row types for ordinary/partitioned tables, views, and materialized views but skips relkind = 'f' even though foreign tables are introspected and emitted elsewhere. A schema with an RPC accepting a foreign-table row, or any metadata needing that relation row type, will not get that type in types, so relationTypeByIds/argument resolution cannot recognize it and generated RPC types fall back to unknown or get filtered. Add f to the included relkinds.
Useful? React with 👍 / 👎.
| json: "Json[Any]", | ||
| jsonb: "Json[Any]", |
There was a problem hiding this comment.
For json/jsonb columns, PostgREST/Supabase Python clients normally hand model code already-decoded JSON objects, arrays, and scalars from the response body, but Pydantic's Json[Any] validates JSON-encoded strings/bytes. Any generated BaseModel for a row with a JSON column therefore rejects normal decoded row data; use Any or a recursive JSON alias for row values instead of Json[Any].
Useful? React with 👍 / 👎.
| return name | ||
| .split(/[^a-zA-Z0-9]+/) // Split on non-alphanumeric characters (like spaces, dashes, etc.) | ||
| .map((word) => word.toLowerCase()) // Convert each word to lowercase | ||
| .join("_"); // Join with underscores |
There was a problem hiding this comment.
When a valid quoted PostgreSQL column name is a Python keyword such as class or from, this formatter returns the keyword unchanged, and the serializers emit invalid class or TypedDict syntax like class: ... = Field(alias="class"). Keep the alias as the database name, but guard the generated Python identifier with a safe suffix or escape strategy.
Useful? React with 👍 / 👎.
| return "unknown"; | ||
| } | ||
|
|
||
| const viewRowTypes = views.filter((view) => view.name === pgType); |
There was a problem hiding this comment.
When a column or RPC argument uses a materialized view's row type, the metadata already emits that relation under generated Views, but this resolver searches only ordinary views and falls through to unknown. Include materializedViews in the row-type lookup so materialized-view row types resolve to the generated Database[...]["Views"][...]["Row"] shape.
Useful? React with 👍 / 👎.
| if (name in this.types) { | ||
| const type = this.types[name]; | ||
| const schema = type!.schema; | ||
| return `${formatForPyClassName(schema)}${formatForPyClassName(type.name)}`; |
There was a problem hiding this comment.
For user-defined types that are present in types but are neither enums nor composites (for example CREATE TYPE ... AS RANGE columns, or domains inside composite attributes), this branch returns a generated class name even though the generator only emits enum aliases and composite BaseModel classes. The resulting module contains annotations like PublicPriceRange with no corresponding definition, so importing or type-checking the generated Python fails; fall back to Any or resolve only the type kinds that are actually emitted.
Useful? React with 👍 / 👎.
Re: second Codex pass (commit
|
…spect Dropping the per-table relationships aggregation also removed TABLES_SQL's top-level GROUP BY, which had incidentally emitted tables in c.oid order via the aggregate plan. Without it the query had no explicit ordering, so rows came back in heap-scan order — environment- and data-dependent. The TypeScript generator sorts objects alphabetically (unaffected), but Go/Python/Swift emit in metadata order, so their output and postgres-meta's order-sensitive typegen snapshots broke depending on the database. Add `order by c.oid` to TABLES_SQL, reproducing the original ordering deterministically. Generator content is unchanged. Verified empirically against postgres-meta's test fixtures: no ORDER BY (heap): users todos users_audit empty table_with_primary_key_other_than_id user_details table_with_other_tables_row_type ... ORDER BY c.oid: users todos users_audit user_details empty table_with_other_tables_row_type table_with_primary_key_other_than_id ... (== expected snapshot order) RED (before fix): sql.test.ts TABLES_SQL snapshot missing `order by c.oid`. The package parity test passed even without ORDER BY because its container happened to return heap order == oid order; postgres-meta CI's DB did not, which is what surfaced the regression (postgres-meta#1084). Tests: - sql.test.ts: TABLES_SQL snapshot now pins `order by c.oid`. - introspect.test.ts: asserts tables are returned in ascending id order. - Kaizen: documented the explicit-ORDER-BY requirement in the package CLAUDE.md. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f9c14a8b6f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ), | ||
| ), | ||
| ...schemaFunctions | ||
| .filter(({ fn }) => fn.argument_types === table.name) |
There was a problem hiding this comment.
For computed fields in a non-public schema, pg_get_function_arguments() can return a schema-qualified row type such as api.users, while table.name is only users; this string comparison then drops the computed field from Row even though PostgREST exposes it. Matching the single input arg's type/relation id (or otherwise normalizing the qualification) would keep computed fields for included schemas outside the search path.
Useful? React with 👍 / 👎.
| const py_type = this.nullable | ||
| ? `Optional[${this.py_type.serialize()}]` | ||
| : this.py_type.serialize(); | ||
| return ` ${this.name}: ${py_type} = Field(alias="${this.pg_name}")`; |
There was a problem hiding this comment.
When a quoted PostgreSQL column name contains " or \, interpolating this.pg_name directly into Field(alias="...") produces invalid Python or a different alias; the same raw alias is used for insert/update TypedDict fields. Valid identifiers like CREATE TABLE t ("has\"quote" text) should still generate importable Python, so the alias string needs Python escaping rather than direct interpolation.
Useful? React with 👍 / 👎.
| return SWIFT_KEYWORDS.includes(propertyName) | ||
| ? `\`${propertyName}\`` | ||
| : propertyName; |
There was a problem hiding this comment.
This keyword guard only handles in, default, and case, so a valid quoted column name such as class, struct, or let is emitted as let class: ... instead of a backticked identifier and the generated Swift fails to compile. The escape list should cover Swift's reserved words (or use a general identifier sanitizer) before returning the property name.
Useful? React with 👍 / 👎.
| nullable = column.is_nullable; | ||
| } | ||
| return [ | ||
| formatForGoTypeName(column.name), |
There was a problem hiding this comment.
This formatted column name is emitted directly as a Go struct field, but formatForGoTypeName() can return identifiers that start with a digit (or even an empty string after sanitizing punctuation). A valid quoted column like "2fa" therefore generates a field such as 2fa string, which makes the Go output uncompilable; the database name can stay in the JSON tag, but the Go identifier needs a safe prefix/fallback.
Useful? React with 👍 / 👎.
| view, | ||
| columnsByTableId[view.id], | ||
| types, | ||
| ["Select"], |
There was a problem hiding this comment.
When a view has is_updatable: true, PostgREST accepts inserts and updates through it, and the TypeScript generator already emits view Insert/Update shapes for that case. The Go generator always passes only Select here, so clients using an updatable view get no typed request structs for valid write operations; include Insert/Update when view.is_updatable is true.
Useful? React with 👍 / 👎.
| (type) => type.attributes.length > 0 && schemasNames.has(type.schema), | ||
| ) | ||
| .map((type) => ctx.typeToClass(type)); | ||
| const py_views = views.map((view) => ctx.viewToClass(view)); |
There was a problem hiding this comment.
This maps every view to only a read BaseModel, even when PostgresView.is_updatable is true. Since PostgREST can insert/update through updatable views, Python users get no TypedDict shapes for valid writes while tables and TypeScript views do, so viewToClass should also produce insert/update dictionaries for updatable views.
Useful? React with 👍 / 👎.
| ), | ||
| ...schemaViews.flatMap((view) => | ||
| generateStruct( | ||
| pgTypeToSwiftStruct(view, columnsByTableId[view.id], "Select", { |
There was a problem hiding this comment.
For every view this path generates only the Select struct, ignoring view.is_updatable. Updatable views are valid PostgREST write targets and TypeScript emits Insert/Update types for them, so Swift clients cannot type valid inserts or updates through those views unless this branches on is_updatable and emits the write models too.
Useful? React with 👍 / 👎.
| : this.py_type.serialize(); | ||
| const annotation = `Annotated[${py_type}, Field(alias="${this.pg_name}")]`; | ||
| const rhs = this.not_required ? `NotRequired[${annotation}]` : annotation; | ||
| return ` ${this.name}: ${rhs}`; |
There was a problem hiding this comment.
For Insert/Update TypedDicts, Field(alias=...) is only annotation metadata; it does not change the actual typed dictionary key. When a quoted column is Foo or victory road, this emits foo/victory_road as the key users are expected to send, so typed payloads use names PostgREST does not recognize; use a functional TypedDict with the original database keys or another alias-aware write model for renamed fields.
Useful? React with 👍 / 👎.
|
|
||
| // Case 2: returns a relation's row type. | ||
| const relation = | ||
| introspectionBySchema[schema.name]?.tables.find( |
There was a problem hiding this comment.
When a function in schema api returns public.users and api.users also exists, this schema-local relation lookup misses the actual return_type_relation_id; the later name-only fallback resolves users against the function schema and emits the wrong row type. Search the table/view collections by relation id instead of limiting the lookup to the function's schema.
Useful? React with 👍 / 👎.
| } | ||
|
|
||
| let output = [ | ||
| `${ident(level)}${accessControl} struct ${struct.formattedStructName}${generateProtocolConformances( |
There was a problem hiding this comment.
With accessControl: "public", this emits public structs but relies on Swift's synthesized memberwise initializer, which remains internal. If the generated file is shipped from a Swift package or another module, downstream code cannot construct the public ...Insert/...Update values even though their properties are public; emit an explicit public initializer for public models.
Useful? React with 👍 / 👎.
… pass Replaces the SQL-level table ordering (reverted) with a single, generator-agnostic sort pass. The Go/Python/Swift generators emit objects in GeneratorMetadata order, so output was sensitive to however the producer ordered its rows — the SQL introspector returns rows in environment-dependent heap order, which is what broke postgres-meta's order-sensitive snapshots after the relationships-aggregation removal dropped TABLES_SQL's incidental GROUP BY ordering. Rather than depend on introspection query order at all, add `sortGeneratorMetadata` (src/sort.ts): a pure pass that canonically orders every collection (relations/functions/types by oid, columns by table_id + ordinal_position, relationships by a stable key), leaving nested semantically-ordered arrays (args, attributes, enum values) untouched. Callers apply it after introspection and before generation; generators document that they expect pre-sorted input and do not enforce ordering themselves. Introspection SQL no longer carries any ORDER BY. Generator content is unchanged; the package parity goldens are byte-identical (the package's test DB already returned oid order, so the sorted output matches). postgres-meta adopts the same pass in its adapter and regenerates its typegen snapshots to the canonical order. - src/sort.ts + exports from `.` and `./generation` - generator JSDoc documents the pre-sort contract - parity test applies sortGeneratorMetadata (goldens unchanged) - unit tests: sort by keys, idempotent, no input mutation, nested order preserved - Kaizen: CLAUDE.md updated to mandate the sort pass over query-order reliance Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
The Go/Python/Swift generators emit objects in GeneratorMetadata order, so output depended on the order introspection returned rows (environment-dependent heap order). Apply the package's new sortGeneratorMetadata pass in the getGeneratorMetadata adapter so all four generators receive canonically-ordered metadata. Regenerate the typegen go/python snapshots accordingly: only ordering changes (the `a_view` view moves to its canonical oid position); struct/class contents are byte-identical. TypeScript and Swift sort internally and are unaffected. Requires @supabase/postgrest-typegen with sortGeneratorMetadata (supabase/pg-toolbelt#302). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
Re: third Codex pass (commit
|
There was a problem hiding this comment.
💡 Codex Review
Sort metadata by semantic keys
When two equivalent databases create the same objects in different orders, their OIDs differ, so this comparator still makes sortGeneratorMetadata() order schemas/tables/views/functions/types differently even though the generated API should be byte-stable for the same schema. Because Go/Python/Swift emit declarations in metadata order, the advertised stabilizer can still produce noisy output churn across environments; sort these collections by semantic keys such as schema/name (with ids only as a tie-breaker) instead of by OID alone.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | keyof DefaultSchema["Tables"] | ||
| | { schema: keyof DatabaseWithoutInternals }, |
There was a problem hiding this comment.
For schemas with an updatable view, the generator emits Insert/Update shapes under Views, but the TablesInsert/TablesUpdate helpers only accept keys from DefaultSchema["Tables"] and the schema-qualified branch does the same. That means TablesInsert<"my_updatable_view"> is rejected or resolves to never even though PostgREST accepts writes through that view and the generated Database[...]["Views"][...]["Insert"] type exists. Include view entries that expose write shapes, or provide corresponding view write helpers, so the public helper API can reach the types being emitted.
Useful? React with 👍 / 👎.
| } | ||
| } | ||
|
|
||
| return `${returnType}${fn.is_set_returning_function && returnsMultipleRows ? "[]" : ""} |
There was a problem hiding this comment.
For a set-returning function declared with ROWS 1, prorows is 1, so this condition omits the [] even though PostgREST still serializes set-returning RPC results as arrays; the row estimate should only influence relationship cardinality metadata such as isOneToOne. Generate the RPC Returns type as an array whenever fn.is_set_returning_function is true, otherwise RETURNS SETOF ... ROWS 1 is typed as a single object/scalar while clients receive an array.
Useful? React with 👍 / 👎.
| const type = typesById.get(fn.return_type_id); | ||
| if (type) { | ||
| return pgTypeToTsType(schema, type.name, { |
There was a problem hiding this comment.
When an RPC argument or return type is a domain, typesById resolves the domain row and this passes the domain name (for example email) into pgTypeToTsType; because the type metadata does not carry typbasetype and domains have no enum values or attributes, that falls through to unknown. Table columns over the same domain are unwrapped by COLUMNS_SQL, so RPCs that accept or return domain types lose their concrete TypeScript type unless the base type is carried through and resolved here.
Useful? React with 👍 / 👎.
sortGeneratorMetadata previously ordered collections by oid (id). Oids are not stable across equivalent databases — the same logical schema created in a different order yields different oids — so an oid sort still produced environment-dependent ordering churn in the Go/Python/Swift output, defeating the stabilizer's purpose (raised in Codex review). Sort by semantic keys instead: schema + name for relations/types, schema + table + ordinal_position for columns, schema + name + identity_argument_types for (overloadable) functions; oid kept only as a final tie-breaker. Output is now byte-stable for the same logical schema regardless of producer/environment. Regenerated the Go/Python parity goldens to the new name-based order (pure reorder, byte-identical content); TypeScript/Swift goldens unchanged (they sort internally). Updated sort.test.ts to use disagreeing ids/names so it proves the sort is semantic, not oid-based. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
Follow-up to the sortGeneratorMetadata semantic-key change (supabase/pg-toolbelt#302): the canonical order is now schema+name based, so the Go/Python typegen snapshots are regenerated to alphabetical order. Pure reorder — struct/class contents are byte-identical. TypeScript/Swift sort internally and are unaffected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1aa77ebccc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ([formattedName, type, name]) => { | ||
| return ` ${formattedName.padEnd(maxFormattedNameLength)} ${type.padEnd( | ||
| maxTypeLength, | ||
| )} \`json:"${name}"\``; |
There was a problem hiding this comment.
When a valid quoted PostgreSQL column name contains a backtick, this raw struct-tag interpolation terminates the tag early and the generated Go file does not parse; names containing " also produce a malformed json tag. Escape or quote the full tag value before emitting it so arbitrary database identifiers remain valid Go source.
Useful? React with 👍 / 👎.
| nullable = | ||
| column.is_nullable || | ||
| column.is_identity || | ||
| column.is_generated || | ||
| !!column.default_value; |
There was a problem hiding this comment.
For a table with a stored GENERATED ALWAYS AS (...) column, this only makes the Swift Insert property optional; the field is still emitted as a writable member. If callers set that property, PostgreSQL/PostgREST rejects the insert, so generated columns should be omitted from insert models or made uninhabitable rather than advertised as valid inputs.
Useful? React with 👍 / 👎.
| const enumType = | ||
| enumTypes.find((type) => type.schema === schema.name) || enumTypes[0]; |
There was a problem hiding this comment.
When an included table column uses an enum from another included schema and the table's own schema has an enum with the same name, COLUMNS_SQL only supplies the unqualified type name, and this lookup prefers the current schema. A column declared as other.status in public.t is therefore generated as Database["public"]["Enums"]["status"] whenever public.status also exists, so the TypeScript type accepts the wrong values; carry the column type OID/schema through metadata instead of resolving by name alone.
Useful? React with 👍 / 👎.
| }) ?? []; | ||
|
|
||
| return { | ||
| formattedStructName: `${formatForSwiftTypeName(table.name)}${operation}`, |
There was a problem hiding this comment.
For valid quoted relations whose names normalize to the same Swift identifier, such as foo-bar and foo_bar in the same schema, both Select/Insert/Update models get the same FooBar... name and are emitted in the same schema enum. The generated Swift then fails with an invalid redeclaration, so the formatter needs a collision-avoidance strategy that preserves distinct database identifiers.
Useful? React with 👍 / 👎.
| if (identity && identity.formattedAttributeName !== "id") { | ||
| output.push( | ||
| `${ident(level + 1)}${accessControl} var id: ${identity.formattedType} { ${identity.formattedAttributeName} }`, |
There was a problem hiding this comment.
When a table has an identity column whose generated property is not id and also has another column that does generate to id, this adds a computed id while the stored let id is emitted below. That valid schema produces an invalid redeclaration in every generated Swift model for the table; check for an existing id property before adding the Identifiable shim.
Useful? React with 👍 / 👎.
… truth for ordering Replicate the TypeScript generator's internal ordering in `sortGeneratorMetadata` (relationships by fk_name/referenced_relation/columns, columns by name within table, function args by name) and remove the now- redundant per-collection sorts from all four generators: - typescript: removed schemas/relationships/columns/args sorts. - go/python: removed the columns-by-name sort. - swift: removed columns/schemas/enums/composite-types sorts. Generators now consume the single canonical pass. Output is byte-identical for ALL generators (parity goldens unchanged) — TypeScript is the source of truth, so its output is pinned; Go/Python/Swift already emitted the same name-order. Two classes of sort intentionally stay inside the generators because the single-collection pass cannot express them: - cross-collection MERGE sorts: TypeScript and Swift merge tables + foreign tables (and views + materialized views) into one per-schema group and sort that by name. Removing these reordered foreign tables / matviews, so they're restored. - TypeScript overload-resolution sorts (conflict detection + overload group rendering), which are algorithmic, not input ordering. Generation unit tests now call generators through a thin `sortGeneratorMetadata` wrapper, mirroring real usage (the caller sorts). Refs PGMETA-120. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 11ff444701
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ${tables | ||
| .filter((table) => schemas.some((schema) => schema.name === table.schema)) |
There was a problem hiding this comment.
When introspect() returns foreignTables, this generator never reads that collection, so a schema exposing a foreign table gets no Go Select/write structs for that valid PostgREST resource. The TypeScript and Swift paths treat foreign tables as table-like, so Go clients lose typed access unless foreignTables are included alongside tables here.
Useful? React with 👍 / 👎.
| const ctx = new PythonContext(types, columns, schemas); | ||
| // Used for efficient lookup of types by schema name | ||
| const schemasNames = new Set(schemas.map((schema) => schema.name)); | ||
| const py_tables = tables.flatMap((table) => { |
There was a problem hiding this comment.
When metadata contains a foreign table, this only maps ordinary tables, so Python output omits the corresponding BaseModel/TypedDict classes even though the introspector populates foreignTables and PostgREST can expose them. Include foreignTables in the table-like generation path so schemas with FDW-backed resources are typed.
Useful? React with 👍 / 👎.
| if (enumType) { | ||
| swiftType = `${formatForSwiftTypeName(enumType.name)}`; |
There was a problem hiding this comment.
When a column's format is an enum from another included schema, the enum is emitted under that schema enum (for example SharedSchema.Status), but this returns only Status inside the table's schema. That unqualified reference is unresolved unless an unrelated local enum with the same name exists, in which case it binds to the wrong type; use the enum's schema when emitting Swift type references.
Useful? React with 👍 / 👎.
| relationship.schema === object.schema && | ||
| relationship.referenced_schema === object.schema && |
There was a problem hiding this comment.
When both schemas are included and a relation has an FK to a table in another schema, this predicate drops the relationship because referenced_schema !== object.schema. The metadata already carries referenced_schema, but the generated Relationships array silently omits these valid PostgREST relationships, so clients lose embed typing for cross-schema FKs.
Useful? React with 👍 / 👎.
| const attributeEntries: [string, string, string][] = | ||
| typeWithRetrievedAttributes.attributes.map((attribute) => [ | ||
| formatForGoTypeName(attribute.name), | ||
| pgTypeToGoType(attribute.type!.format, false), |
There was a problem hiding this comment.
For a composite type attribute whose type is a user enum, this call omits the types collection, so pgTypeToGoType() cannot detect the enum and falls back to interface{}. Tables pass the same context correctly; composite structs should do so too or fields like status user_status are generated less precisely than their table-column equivalents.
Useful? React with 👍 / 👎.
| } else if (compositeTypes) { | ||
| // Append a `Select` to the composite type, as that is how is named in the generated struct. | ||
| swiftType = `${formatForSwiftTypeName(compositeTypes.name)}Select`; |
There was a problem hiding this comment.
When a table column uses a composite type, the generator emits that composite as Address, but this branch returns AddressSelect, which only exists for table/view row structs. Schemas with composite-typed columns therefore produce Swift that does not compile; only append Select for relation row types, not emitted composite types.
Useful? React with 👍 / 👎.
| const attributeEntries: PythonBaseModelAttr[] = attributes.map( | ||
| (attribute) => { | ||
| const type = this.parsePgType(attribute.type!.name); | ||
| return new PythonBaseModelAttr(attribute.name, type, false); |
There was a problem hiding this comment.
PostgreSQL composite type attributes can be null, but every generated Python composite model field is marked non-optional here. If an RPC or composite-typed column returns a value like (NULL, 'city')::address, Pydantic rejects the valid response; composite attributes should be emitted as optional/nullable unless the metadata carries a real not-null constraint.
Useful? React with 👍 / 👎.
| typeWithRetrievedAttributes.attributes.map((attribute) => { | ||
| return { | ||
| formattedAttributeName: formatForSwiftTypeName(attribute.name), | ||
| formattedType: pgTypeToSwiftType(attribute.type!.format, false, { |
There was a problem hiding this comment.
PostgreSQL composite type attributes can contain nulls, but composite structs force every attribute through pgTypeToSwiftType(..., false). A valid composite value with a null field then fails Codable decoding because the generated property is non-optional; composite attributes should be nullable unless the introspected metadata proves otherwise.
Useful? React with 👍 / 👎.
2 participants
Summary
Introduces
@supabase/postgrest-typegen, a new package that provides type generation for PostgREST from PostgreSQL schemas. This is the type-generation engine extracted from postgres-meta, offering a clean separation between introspection and code generation with support for TypeScript, Go, Python, and Swift.Key Changes
packages/postgrest-typegen/with complete introspection and generation pipelineDatabasetype with Row/Insert/Update operations, relationships, and helper typesGeneratorMetadatacontract, ensuring compile-time equivalence with postgres-meta's types00-init.sql,01-memes.sql) providing test data including tables, views, enums, functions, and relationshipsNotable Implementation Details
GeneratorMetadatais the central interface allowing different introspection sources (SQL, API, etc.) to feed the same generatorsdetectOneToOneRelationships,postgrestVersion,defaultSchemafor TypeScript;accessControlfor Swift)Testing
bun run test:postgrest-typegenhttps://claude.ai/code/session_01B53a2ayu9d16u821FQ5Qqz