Skip to content

Custom serialization#61

Open
danieroux wants to merge 7 commits intoonyx-platform:masterfrom
danieroux:custom-serialization-issue-60
Open

Custom serialization#61
danieroux wants to merge 7 commits intoonyx-platform:masterfrom
danieroux:custom-serialization-issue-60

Conversation

@danieroux
Copy link
Copy Markdown

This implements #60

I made the configs first-class keys, and not using kafka/consumer-opts. Consumer-opts feels more like "whatever this plugin did not consider".

Copy link
Copy Markdown
Member

@lbradstreet lbradstreet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks good to merge once my minor comments are addressed.

:default 2000
:optional? true}

:kafka/deserializer
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:kafka/deserializer -> :kafka/value-deserializer for consistency with kafka.

Copy link
Copy Markdown
Author

@danieroux danieroux Nov 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is a tricky one for me.

I was keeping it consistent with :kafka/deserializer-fn - if you change the :kafka/deserializer you have to consider how it affects your :kafka/deserializer-fnchoice.

I changed the documentation to say something about it.

Do you still want me to change it to be consistent with kafka?

Comment thread src/onyx/kafka/information_model.cljc Outdated
Comment thread test/onyx/plugin/serialization_test.clj
…ray is expected

- Also assert in the test that the key/value in those functions are what we expect
- Preserve take-now signature before custom serializers were added
- Allow end-offsets and beginning-end-offsets-clj to be used with custom serializers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants