Skip to content

Conversation

@rapimo
Copy link
Member

@rapimo rapimo commented Jul 3, 2018

this adds two new scan methods

  • raw simply dumps the message into a single column with only using the columns type in function
  • custom allows the user to define a custom decoding function / dynamic lib which will be automatically loaded, the function kafka_custom_msglen is added here as a rather useless example

this also add the experimental expandKafkaScanPData methods which would split the expandKafkaScanPData into chunks of batch size allowing multiple parallel workers to do the job better, but this seems not to be very fruitfull

rapimo added 7 commits July 3, 2018 09:39
this might be bebefitial for parallel scans as worker might be
better balances.
However improvemt in useless kafkaNext calls when subsequent KafkaScanP
are close to each other might be needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants