This repository has been archived by the owner on Mar 24, 2021. It is now read-only.
Releases: Parsely/pykafka
Releases · Parsely/pykafka
2.3.0
Minor Version Features
- Added the
ManagedBalancedConsumer
class, which performs balanced consumption
using the Kafka 0.9 Group Membership API - Added the
managed
keyword argument toTopic.get_balanced_consumer
to access
ManagedBalancedConsumer
- Added a
compacted_topic
kwarg toBalancedConsumer
to make it smarter about
offset ordering for compacted topics - Added methods to
Broker
that use the Group Membership API - Changed the terminology "offset manager" to "group coordinator" to match updated
Kafka jargon - Added new exception types from Kafka 0.9
- Added
auto_start
keyword argument toProducer
to match the consumer interface - Added
max_request_size
keyword argument toProducer
to catch large messages
before they're sent to Kafka - Added protocol functions for the Group Membership API
- New
SimpleConsumer
keyword arguments:compacted_topic
,generation_id
,
consumer_id
Bug Fixes
- Fixed a bug in Travis config causing tests not to run against Kafka 0.9
- Upgraded to non-beta gevent version
- Allowed a single
Broker
instance to maintain multiple connections to a broker
(useful when multiple consumers are sharing the sameKafkaClient
) - Allowed switchable socket implementations when using gevent
- Handled
TypeError
during worker thread shutdown to avoid nuisance messages - Limited
Producer.min_queued_messages
to 1 whensync=True
- Monkeypatched a bug in py.test causing tests to be erroneously skipped
Miscellaneous
- Added an issue template
2.2.1
Bug Fixes
- Fixed Travis issues related to PyPy testing
- Fixed deprecated dependency on gevent.coros
- Enabled caching in Travis for pip, librdkafka, and kafka installations
- Fixed a crash during metadata updating related to zookeeper fallback
- Unified connection retry logic in
Cluster
- Raised an exception if consumer offset reset fails after maximum retries
- Fixed a bug allowing
get_delivery_report
to indefinitely blockproduce()
- Fixed a bug causing producers to drop
to_retry
messages onstop()
- Added retry logic to offset limit fetching
2.2.0
Minor Version Features
- Added support for gevent-based concurrency in pure cpython
- Allowed ZooKeeper hosts to be specified directly to KafkaClient instead of
being treated as a fallback
Bug Fixes
- Fixed a bug causing
RLock
-related crashes in Python 3 - Used the more stable sha1 hash function as the default for
hashing_partitioner
- Fixed a bug in the meaning of
linger_ms
in the producer
2.1.2
Features
- Allowed consumers to run with no partitions
Bug Fixes
- Fixed a bug causing consumers to hold outdated partition sets
- Handled some previously uncaught error codes in
SimpleConsumer
- Fixed an off-by-one bug in message set fetching
- Made
consume()
stricter about message ordering and duplication
2.1.1
Features
- Improved unicode-related error reporting in several components
- Removed the ZooKeeper checker thread from the
BalancedConsumer
- Added a test consumer CLI to
kafka_tools
Bug Fixes
- Fixed a memory leak in the rdkafka-based consumer
- Fixed offset committing to work against Kafka 0.9
- Improved the reliability of the Kafka test harness
Miscellaneous
- Simplified the Travis test matrix to handle testing against multiple Kafka versions
2.1.0
Features
- Addded an optional C extension making use of librdkafka for enhanced producer and
consumer performance - Added a delivery report queue to the
Producer
allowing per-message errors
to be handled - Added a callback indicating that the
BalancedConsumer
is in the process of rebalancing
Bug Fixes
- Fixed a longstanding issue causing certain tests to hang on Travis
- Fixed a bug causing the default error handles in the consumer to mask unknown error
codes - Moved the
Message
class to using__slots__
to minimize its memory footprint
2.0.4
Features
- Allowed discovery of Kafka brokers via a ZooKeeper connect string supplied to
KafkaClient
Bug Fixes
- Made
BalancedConsumer
's ZooKeeper watches close quietly on consumer exit - Disconnect sockets in response to any socket-level errors
- Fixed
HashingPartitioner
for python 3
2.0.3
Features
- Raise exceptions from worker threads to the main thread in
BalancedConsumer
- Call
stop()
whenBalancedConsumer
is finalized to minimize zombie threads
Bug Fixes
- Use weak references in
BalancedConsumer
workers to avoid zombie threads creating
memory leaks - Stabilize
BalancedConsumer.start()
- Fix a bug in
TopicDict.values()
causing topics to be listed asNone
- Handle
IOError
inBrokerConnection
andsocket.recvall_into
- Unconditionally update partitions' leaders after metadata requests
- Fix thread-related memory leaks in
Producer
- Handle connection errors during offset commits
- Fix an interpreter error in
SimpleConsumer
2.0.2
Features
- Switched the
BalancedConsumer
to using ZooKeeper as the single source of truth
about which partitions are held - Made
BalancedConsumer
resilient to ZooKeeper failure - Made the consumer resilient to broker failure
Bug Fixes
- Fixed a bug in
BrokerConnection
causing the message length field to
occasionally be corrupted - Fixed a bug causing
RequestHandler
worker threads to sometimes abort
before the request was completed - Fixed a bug causing
SimpleConsumer
to hang when the number of brokers in
the cluster goes below the replication factor
2.0.1
Features
- Added support for python 3.5
- Added iteration to the
BalancedConsumer
- Disallowed
min_queued_messages<1
inProducer
- Made
SimpleConsumer
commit offsets onstop()
- Supported
None
in message values
Bug Fixes
- Improved
BalancedConsumer
's handling of an externalKazooClient
instance - Fixed
kafka_tools.py
for Python 3 - Removed the unused
use_greenlets
kwarg fromKafkaClient
- Improved
Cluster
's ability to reconnect to brokers during metadata updates - Fixed an interpreter error in
conncection.py
- Fixed failure case in
Producer
whenrequired_acks==0
- Fixed a bug causing
SimpleConsumer
to leave zombie threads after disconnected brokers - Improved
SimpleConsumer
's worker thread exception reporting - Simplified
SimpleConsumer
's partition locking logic duringfetch
by usingRLock
- Fixed
SimpleConsumer
off-by-one error causing lag to never reach 0
Miscellaneous
- Switched from Coveralls to Codecov for converage tracking