Skip to content
This repository has been archived by the owner on Oct 7, 2022. It is now read-only.

Synchronous flush #8

Open
rhnasc opened this issue Aug 28, 2018 · 0 comments · May be fixed by #84
Open

Synchronous flush #8

rhnasc opened this issue Aug 28, 2018 · 0 comments · May be fixed by #84

Comments

@rhnasc
Copy link

rhnasc commented Aug 28, 2018

KAFKA_CONSUMER_BATCH_SIZE controls when the in-memory events are flushed to ES.

However some topics may have small throughput, not having their events flushed at all.

It would be nice to have a second parameter controlling a recurrent tick so the in-memory events are flushed even when they do not meet the batch size requirement
@ggml1 ggml1 linked a pull request Apr 23, 2021 that will close this issue
Open
5 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add deadline on consumer to send non-empty record batch inloco/kafka-elasticsearch-injector
1 participant
@rhnasc