Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

making bulkChannel buffer size configurable? #271

Open
Dieterbe opened this issue Apr 16, 2016 · 0 comments
Open

making bulkChannel buffer size configurable? #271

Dieterbe opened this issue Apr 16, 2016 · 0 comments

Comments

@Dieterbe
Copy link

bulkindexer exposes some config options to tune when to flush etc, but it's currently hardcoded (via bulkChannel and StartDocChannel) that if we're busy flushing, we only allow up 100 new items before any call such as BulkIndexer.Index() starts blocking.

we have an application where we may want to add millions of items at once, and if it takes a while for all that data to be bulkindexed, that's fine, but we don't want our app to be hanging on Index() calls.

would it make sense to add such a buffer to elastigo, or should i do it in my application?
a configurable bulkChannel size would be a good start but that allocates all the space up front, so a buffer that can grow (up to the limit) and shrink as needed might be even better ( though i'm not sure how to implement that elegantly, it may not be worth it.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant