postgres_cdc: relax aws.endpoint to accept bare hostnames#4434
Open
david-yu wants to merge 2 commits into
Open
postgres_cdc: relax aws.endpoint to accept bare hostnames#4434david-yu wants to merge 2 commits into
david-yu wants to merge 2 commits into
Conversation
The AWS SDK's `auth.BuildAuthToken` rejects an endpoint without a port
("the provided endpoint is missing a port, or the provided port is
invalid"), but `aws.endpoint` was documented as a bare hostname. Mirror
the mysql_cdc fix from redpanda-data#4433: derive the port from the parsed pgconn
config when the user omits it (falling back to 5432), and fall back to
the DSN address entirely when `aws.endpoint` is empty. Field is now
optional with a `""` default.
Backwards compatible: explicit `host:port` values are still passed
through unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The S3 read benchmark added in redpanda-data#4369 (commit bed70d9) is failing the modernize linter on main: `waitgroup` wants WaitGroup.Go on line 141, `minmax` wants max() on line 200. Both fixes are mechanical and unrelated to mysql_cdc, included here so this PR's golangci-lint job can be green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #4433 (mysql_cdc). The same
aws.endpointpapercut exists onpostgres_cdc: the AWS SDK's RDS token signer requireshost:portbut the field is documented (and rendered in--help) as a bare hostname. Users hitting it see:This PR mirrors the mysql_cdc fix:
internal/impl/postgresql/input_pg_stream.go— endpoint description updated to call outhost:portand the auto-derive behaviour. Field is now optional with a""default.internal/impl/postgresql/aws/aws.go— newnormalizeIAMEndpoint(endpoint, host, port)helper. If endpoint is empty, usepgconn.Config.Host+Port; if it's a bare hostname, splice in the DSN port (5432fallback).internal/impl/postgresql/aws/aws_test.go— table-driven unit tests for the helper.docs/modules/components/pages/inputs/postgres_cdc.adoc— regenerated viatask docs.Only N7 from the operator PR thread applies to postgres_cdc:
AllowCleartextPasswords) is MySQL-driver-specific; pgx accepts the IAM token natively.REQUIRE SSL) is MySQL syntax.UX before / after
Before — bare hostname produces a confusing SDK error:
After — the port from the DSN is spliced in (or 5432 as a final fallback).
aws.endpointcan also be omitted entirely:Backwards compatibility
Fully compatible.
host:portvalues are returned unchanged; the new fallbacks only fire when the user supplied a value that previously errored out.aws.endpoint: "host:5432"aws.endpoint: "host"(bare hostname)aws.endpointomittedaws.enabled: false(or unset)Field schema goes from required to optional with
""default — a relaxation; nothing that previously parsed will now fail.Test plan
go test ./internal/impl/postgresql/...golangci-lint run ./internal/impl/postgresql/...task docs(regenerated postgres_cdc.adoc)aws.enabled: trueand a bare-hostnameaws.endpoint🤖 Generated with Claude Code