Broker interceptor #3
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
October 18, 2017 14:46
|
|
tozhan-zz
reviewed
Oct 19, 2017
| val clientId = request.header.clientId | ||
|
|
||
| var result = clientId.split('_'); | ||
| var eventPattern = result(1); |
There was a problem hiding this comment.
In Kafka 0.11, you can use Record Header to store eventPattern info.
Author
There was a problem hiding this comment.
Yes, that is my previous plan, but it need additional work to parse it from header. I think this is not the highest priority issue we need to resolve, we could left hack way here and make code work firstly.
tozhan-zz
reviewed
Oct 19, 2017
| } | ||
| } | ||
|
|
||
| while (batchs.hasNext()) { |
Author
There was a problem hiding this comment.
yes, should be. But I test this two separately and add them together, so this is not the issue cause get out key failed.
tozhan-zz
reviewed
Oct 19, 2017
| var eventPatternRegex = eventPattern.r; | ||
| var records = logReadInfo.records.batches().iterator(); | ||
| if (!eventPatternRegex.findFirstIn(key.toString).isDefined) { | ||
| records.remove(); |
There was a problem hiding this comment.
Need we store the final records to the result?
Author
There was a problem hiding this comment.
Need to double check and confirm this thing after resolve get out key failed issue.
kondetibharat
pushed a commit
that referenced
this pull request
Jan 16, 2020
…pache#7305) A partition log in initialized in following steps: 1. Fetch log config from ZK 2. Call LogManager.getOrCreateLog which creates the Log object, then 3. Registers the Log object Step #3 enables Configuration update thread to deliver configuration updates to the log. But if any update arrives between step #1 and #3 then that update is missed. It breaks following use case: 1. Create a topic with default configuration, and immediately after that 2. Update the configuration of topic There is a race condition here and in random cases update made in second step will get dropped. This change fixes it by tracking updates arriving between step #1 and #3 Once a Partition is done initializing log, it checks if it has missed any update. If yes, then the configuration is read from ZK again. Added unit tests to make sure a dirty configuration is refreshed. Tested on local cluster to make sure that topic configuration and updates are handled correctly. Reviewers: Jason Gustafson <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a draft code change, there are a lot of hard code and bug inside.