Skip to content
This repository has been archived by the owner on Mar 24, 2021. It is now read-only.

Support librdkafka in the ManagedBalancedConsumer #778

Open
waytai opened this issue Mar 26, 2018 · 3 comments
Open

Support librdkafka in the ManagedBalancedConsumer #778

waytai opened this issue Mar 26, 2018 · 3 comments

Comments

@waytai
Copy link

waytai commented Mar 26, 2018

Traceback (most recent call last):                                                                          
  File "pyconsume.py", line 30, in <module>                                                                 
    use_rdkafka = True                                                                                      
  File "/usr/local/lib/python2.7/dist-packages/pykafka/topic.py", line 214, in get_balanced_consumer        
    return cls(self, self._cluster, consumer_group, **kwargs)                                               
  File "/usr/local/lib/python2.7/dist-packages/pykafka/managedbalancedconsumer.py", line 188, in __init__   
    self.__class__.__name__))                                                                               
ImportError: use_rdkafka is not available for ManagedBalancedConsumer                                       
Exception AttributeError: "'ManagedBalancedConsumer' object has no attribute '_running'" in <bound method Ma
nagedBalancedConsumer.__del__ of <pykafka.managedbalancedconsumer.ManagedBalancedConsumer at 0x7f5a65a31b10 
(consumer_group=KAFKA_01)>> ignored
@emmettbutler emmettbutler changed the title use_rdkafka = True , and I have installed librdkafka Support librdkafka in the ManagedBalancedConsumer Mar 26, 2018
@emmettbutler
Copy link
Contributor

At the time the ManagedBalancedConsumer was written, librdkafka didn't support Kafka-managed consumer groups, so this ImportError was added to make that clear. We should reexamine where librdkafka's support is these days, and fix this if possible. This will probably happen during Parse.ly's internal upgrade to latest Kafka later this year.

@waytai
Copy link
Author

waytai commented Mar 27, 2018

than you for your kindly reply

@razor-1
Copy link

razor-1 commented Aug 16, 2018

seems like confluentinc/librdkafka#601 (comment) indicates some of how this works, and the issue overall explains that librdkafka supports this now, if I'm reading it correctly

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants