DNM: kinesis support for bridge #924
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a copy/paste from the repo where this work was being done originally.
Embeds a deno runtime in the bridge binary to allow ad hoc reshaping of
messages read from queues before they are sent to Svix.
## Motivation
When consuming JSON from messaging systems to generate new webhooks,
it's common to need to transform the payload before making the Create
Message request to Svix.
## Solution
Embedding a JS runtime into the bridge process allows us to run
user-defined functions on the payloads as they move from input to
output.
With this diff, plugin instances provided by
`svix-webhook-bridge-plugin-queue-consumer` can now evaluate js included
in the plugin config:
```yaml
plugins:
- type: "rabbitmqconsumer"
input:
uri: "amqp://guest:guest@localhost:5672/%2f"
queue_name: "local"
requeue_on_nack: false
transformation: |
function handler(input) {
return {
app_id: input.key,
message: {
eventType: input.event_type,
payload: input.data
}
};
}
output:
token: "***************"
```
The `transformation` key can now be set on any of these consumers. The
JS fragment should include a default export of a function that accepts
an object and returns an object.
These functions can be used to reshape the payload as necessary.
Very incomplete/untested. The sdk doesn't seem to have APIs for consuming. Follow <awslabs/aws-sdk-rust#213> for updates on the sdk side.
svix-onelson
force-pushed
the
onelson/bridge
branch
2 times, most recently
from
c2f47e8 to
e567257
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Do not merge. Only a partial impl.
Needs consumer APIs which are not yet implemented in the SDK awslabs/aws-sdk-rust#213
Much of the code in the new kinesis module is actually copypasta from SQS, so don't be fooled. It probably looks more complete than it is.