feat(webhooks): outbound webhook outbox with dispatcher, retries & admin ops - #75
Open
priscaenoch wants to merge 1 commit into
Open
feat(webhooks): outbound webhook outbox with dispatcher, retries & admin ops#75priscaenoch wants to merge 1 commit into
priscaenoch wants to merge 1 commit into
Conversation
…& admin ops Implements a transactional outbox for platform events (issue MyFanss#54): durable WebhookEvent rows, a dispatcher with exponential backoff, admin inspection, and payload redaction — preparing partner webhooks without a real HTTP delivery client. - New `webhooks` module: WebhookEvent entity/migration (indexed on status+nextAttemptAt), WebhooksService (emit/dispatch/retry/list), admin routes under /api/v1/admin/webhook-events (list, dispatch, retry — admin RBAC only), and a deep payload redactor (passwords/tokens/secrets/emails). - Wired emitters: `subscription.created` on subscribe (new + reactivate) and `post.published` on post creation, both writing the outbox row inside the same DB transaction as the domain write via dataSource.transaction(). - Dispatcher has no cron dependency (none installed); it's triggered via the admin dispatch endpoint. Logs to console by default, or POSTs to WEBHOOK_DEBUG_URL if configured. Failures back off exponentially and go `dead` after 5 attempts; dead events can be retried. - docs/webhooks-outbox.md covers the data model, emitters, redaction, dispatcher and admin API. 20 new tests in src/webhooks/, plus new emitter-specific tests in subscriptions/posts service specs. Also fixes pre-existing breakage on main unrelated to this issue, needed to get build/format/tests green: a duplicate AuditAction.USER_SELF_DELETED enum member, missing AuditAction members referenced by billing-webhooks.service.ts, a missing `deletedAt` field in FeedService's post mapping, and PostsService missing the getArchivedPosts/restorePost methods (plus post.entity's deletedById column) that posts.controller.ts, posts.service.spec.ts and test/posts.e2e-spec.ts already expected from a prior merge. Reformatted 3 previously-unformatted files to pass format:check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the transactional outbox described in #54: durable
WebhookEventrows, a dispatcher with exponential backoff, admin inspection, and payload redaction — preparing partner webhooks without a real HTTP delivery client.closes #54
webhooksmodule:WebhookEvententity + migration (indexed onstatus+nextAttemptAt),WebhooksService(emit/dispatch/retry/listEvents), admin routes under/api/v1/admin/webhook-events(list, dispatch, retry — admin RBAC only,403for non-admins), and a deep payload redactor stripping passwords/tokens/secrets/emails from nested objects/arrays.subscription.createdon subscribe (both new subscriptions and reactivating a cancelled one) andpost.publishedon post creation — both now write viadataSource.transaction()so the outbox row commits atomically with the domain write.POST /api/v1/admin/webhook-events/dispatch, which is also how tests and admins run it on demand. It logs to console by default, or POSTs toWEBHOOK_DEBUG_URLif configured. Failures back off exponentially (capped at 5 min) and godeadafter 5 attempts; dead events can be retried viaPOST .../:id/retry.docs/webhooks-outbox.mddocuments the data model, emitters, redaction rules, dispatcher behavior, and admin API, including what's intentionally out of scope (signed partner delivery, per-creator webhook management UI).src/webhooks/(service, redaction, admin controller) plus emitter-specific tests added to the existingsubscriptions.service.spec.tsandposts.service.spec.ts.Pre-existing baseline breakage (unrelated to #54, fixed to keep CI green)
maincurrently failsnpm run build/test:e2e, apparently from a bad merge of apost-soft-delete-archive-restorefeature where the controller/tests/migration landed but the service implementation didn't:AuditAction.USER_SELF_DELETEDenum member (build-breaking).AuditActionmissing severalBILLING_WEBHOOK_*members thatbilling-webhooks.service.tsalready referenced.FeedService's post mapping was missing the now-requireddeletedAtfield.PostsServicewas missinggetArchivedPosts/restorePost(andPostwas missingdeletedById) even thoughposts.controller.ts,posts.service.spec.ts, andtest/posts.e2e-spec.tsalready called them — reconstructed faithfully against those existing call sites and the already-presentAddPostSoftDeletemigration.test/posts.e2e-spec.ts) used a numeric user id where every other public-route assertion in that file uses the creator'shandle— fixed to match the surrounding pattern.I confirmed with the requester before making these fixes, since they're outside #54's scope but were blocking
build/test/format:checkregardless of this PR's contents.Testing / validation performed
npm run build— clean.npm run format:check— clean.npm test— 49 suites / 380 tests pass.npm run test:e2e(against a local Postgres via Docker, since no DB is available by default) —test/posts.e2e-spec.ts(43 tests) passes in full, including the reconstructed archive/restore endpoints. Did not run the entire e2e/integration suite end-to-end locally; CI's Postgres service should cover the rest.