Skip to content
This repository has been archived by the owner on Mar 24, 2021. It is now read-only.

consumer rate decreasing #791

Closed
arita37 opened this issue Apr 25, 2018 · 1 comment
Closed

consumer rate decreasing #791

arita37 opened this issue Apr 25, 2018 · 1 comment

Comments

@arita37
Copy link

arita37 commented Apr 25, 2018

Hello,

We are using normal code as follow :
#781

consumer = topic.get_balanced_consumer()  # all default parameters...  3 balanced consumer
while True:
    try:
        for message in consumer:
              ii+=1
              msgraw = consumer.consume()
              msg    = msgraw.value.decode('utf8')
              msg    = json.loads(msg)   
              .... #very fast process
              producer.produce(json_data)
   except Exception as e : 
               print(e)

What happens :
0 min - 5 mins : Input flow is same Kafka server (test server with fixed flow).
After 5 mins: Almost half of the expected.

  1. Just wondering if you have done some tests to see the consuming flow is matching kafka output ?

  2. Is 1/2 of the flow matching anything in consumer config ?

Can we re-open this issue ?

@emmettbutler
Copy link
Contributor

Duplicate of #790

@emmettbutler emmettbutler marked this as a duplicate of #790 Apr 25, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants