Skip to content

Consumer

Matt Howlett edited this page Dec 18, 2019 · 23 revisions

Committing Offsets

https://docs.confluent.io/current/clients/dotnet.html#auto-offset-commit

Question: I want to synchronously commit offsets after each consumed message. It's very slow. How do I make it fast?

librdkafka uses the same broker connection both for commits and fetching messages, thus a commit may be backed up behind a long-poll blocking Fetch request. The long-poll behavior of fetch requests is configured with the fetch.wait.max.ms property that defaults to 100ms. You can decrease that value to decrease offset commit latency at the expense of a larger number of Fetch requests (depending on your traffic pattern). Also see https://github.com/edenhill/librdkafka/issues/1787

todo: is this information current?

What are the trade-offs regarding the number of .NET client consumer instances?