Pykafka

.. _Compare 2.8.0-dev.1: https://github.com/Parsely/pykafka/compare/2.7.0...2.8.0-dev.1

Minor Version Features
----------------------

* Added a `deserializer` kwarg to consumer components to facilitate unicode support
* Added a `reset_offset_on_fetch` kwarg to consumer components to support read-only
consumption
* Changed the expected type of the consumer's `consumer_group` kwarg to `str` from `bytes`
* Changed the expected type of `TopicDict.__getitem__`'s parameter to `str` from `bytes`
* Added a `pending_timeout_ms` kwarg to `Producer.__init__` to allow delivery report
wait timeouts
* Added a `serializer` kwarg to `Producer.__init__` to facilitate unicode support
* Deprecated the `generation_id` and `consumer_id` parameters on `SimpleConsumer`

Bugfixes
--------

* Changed consumers to handle valid ascii strings for consumer group names instead of
bytes
* Handled `NoNodeException` during consumer ZK node releases
* Used `api_versions` to select the version-appropriate implementation for
`OffsetFetchRequest`
* Adjusted synchronous production logic to avoid infinite blocking when delivery report
is lost
* Fixed a bug in `FetchResponseV1` causing `throttle_time` to be returned as a tuple
instead of an integer
* Implemented support for all current versions of `OffsetFetchRequest` and
`OffsetFetchResponse`

Miscellaneous
-------------

* Used logging.NullHandler to remove nuisance logs
* Added stock unicode serde to `utils`
* Added README to pypi info

.. _Compare 2.7.0: https://github.com/Parsely/pykafka/compare/2.6.0...2.7.0

Minor Version Features
----------------------

* Added a `broker_version` kwarg to `Broker.__init__` for the purpose of setting
`api_version` in `FetchResponse`
* Added a `topic_name` argument to `Broker.join_group` for use in protocol metadata,
visible via the Administrative API
* Added a function `print_managed_consumer_groups` to the CLI
* Added a `timestamp` kwarg to `Producer.produce` to pass on messages when the broker
supports newer message formats
* Changed `Producer.produce` to return the produced `Message` instance
* Added `protocol_version` and `timestamp` kwargs to `Message`
* Added support for the `fetch_error_backoff_ms` kwarg on `SimpleConsumer`
* Added an `unblock_event` kwarg to `SimpleConsumer.consume` used to notify the consumer
that its parent `BalancedConsumer` is in the process of rebalancing
* Added a general-purpose `cleanup` function to `SimpleConsumer`
* Added a `membership_protocol` kwarg to `BalancedConsumer` that allows switchable and
user-defined membership protocols to be used
* Implemented `GroupMembershipProtocol` objects for the two standard partition assignment
strategies
* Added an `api_versions` kwarg to `Broker` to facilitate switchable API protocol versions
* Added support for all versions of the `MetadataRequest` to `Broker`
* Added the `controller_broker` attribute to `Cluster`
* Added `create_topics` and `delete_topics` to `Broker`
* Added `fetch_api_versions` to `Broker` and `Cluster`
* Added a CLI for creating and deleting topics on the cluster to `kafka_tools`
* Added support for LZ4 compression to the `Producer` and `SimpleConsumer`

Bug Fixes
---------

* Added an `Event` that notifies the internal `SimpleConsumer` of a `BalancedConsumer`
that a rebalance is in progress, fixing a bug causing partitions to be unreleased
* Fixed a bug causing busywaiting in the `BalancedConsumer` when there are no partitions
available
* Updated the protocol implementation to send non-empty `GroupMembershipProtocol`
objects and become compatible with the Administrative API
* Fixed a bytestring bug causing `kafka_tools.reset_offsets` not to work in python 3
* Added a separate retry limit on connections to the offset manager
* Improved logging on socket errors
* Fixed a bug causing API version not to be passed on certain requests
* Handled new `MessageSet` compression scheme in API v1
* Fixed a bug in `rdkafka.SimpleConsumer` causing exceptions not to be raised from worker
threads
* Fixed a bug causing `fetch_offsets` not to raise exceptions under certain conditions
when it should
* Adjusted `Cluster` to become aware of supported API versions immediately upon
instantiation
* Refactored code in `Cluster` related to metadata requests to make logic reusable for
pre-bootstrap communication with the cluster
* Added the ability to pass arguments to `protocol.Response` instances when waiting
on a future
* Adjusted the `RandomPartitioner` to avoid actually calling `random.choice` to improve
performance
* Removed some calls in `Producer.procuce` to `isinstance` to improve performance
* Simplified retry logic in `SimpleConsumer.fetch_offsets`

Miscellaneous
-------------

* Separated gevent tests from other builds in Travis
* Made dependency on gevent optional
* Added a convenient CLI entry point via `__main__`
* Fixed exception naming convention to align with naming in the broker
* Avoided building the `rdkafka` extension on platforms that don't support it
* Fixed a bug in test harness causing some tests not to be inherited from parent classes
* Used `sudo: required` to get around dead Travis machines
* Upgraded Travis tests to use Kafka 1.0.0
* Added Code of Conduct
* Documented release process
* Made PyKafka available via conda-forge
* Fleshed out the beginning of the usage guide
* Made `kafka_instance` fetch its binary from `archive.apache.org` instead of
`mirror.reverse.net` because the latter removed old versions of Kafka

.. _Compare 2.7.0-dev.2: https://github.com/Parsely/pykafka/compare/2.7.0.dev1...2.7.0-dev.2

Minor Version Features
----------------------

* Added a `membership_protocol` kwarg to `BalancedConsumer` that allows switchable and
user-defined membership protocols to be used
* Implemented `GroupMembershipProtocol` objects for the two standard partition assignment
strategies
* Added an `api_versions` kwarg to `Broker` to facilitate switchable API protocol versions
* Added support for all versions of the `MetadataRequest` to `Broker`
* Added the `controller_broker` attribute to `Cluster`
* Added `create_topics` and `delete_topics` to `Broker`
* Added `fetch_api_versions` to `Broker` and `Cluster`
* Added a CLI for creating and deleting topics on the cluster to `kafka_tools`
* Added support for LZ4 compression to the `Producer` and `SimpleConsumer`

Bug Fixes
---------

* Adjusted `Cluster` to become aware of supported API versions immediately upon
instantiation
* Refactored code in `Cluster` related to metadata requests to make logic reusable for
pre-bootstrap communication with the cluster
* Added the ability to pass arguments to `protocol.Response` instances when waiting
on a future
* Adjusted the `RandomPartitioner` to avoid actually calling `random.choice` to improve
performance
* Removed some calls in `Producer.procuce` to `isinstance` to improve performance
* Simplified retry logic in `SimpleConsumer.fetch_offsets`

Miscellaneous
-------------

* Used `sudo: required` to get around dead Travis machines
* Upgraded Travis tests to use Kafka 1.0.0
* Added Code of Conduct
* Documented release process
* Made PyKafka available via conda-forge
* Fleshed out the beginning of the usage guide
* Made `kafka_instance` fetch its binary from `archive.apache.org` instead of
`mirror.reverse.net` because the latter removed old versions of Kafka

.. _Compare 2.7.0.dev1: https://github.com/Parsely/pykafka/compare/2.6.0...2.7.0.dev1

Minor Version Features
----------------------

* Added a `broker_version` kwarg to `Broker.__init__` for the purpose of setting
`api_version` in `FetchResponse`
* Added a `topic_name` argument to `Broker.join_group` for use in protocol metadata,
visible via the Administrative API
* Added a function `print_managed_consumer_groups` to the CLI
* Added a `timestamp` kwarg to `Producer.produce` to pass on messages when the broker
supports newer message formats
* Changed `Producer.produce` to return the produced `Message` instance
* Added `protocol_version` and `timestamp` kwargs to `Message`
* Added support for the `fetch_error_backoff_ms` kwarg on `SimpleConsumer`
* Added an `unblock_event` kwarg to `SimpleConsumer.consume` used to notify the consumer
that its parent `BalancedConsumer` is in the process of rebalancing
* Added a general-purpose `cleanup` function to `SimpleConsumer`

Bug Fixes
---------

* Added an `Event` that notifies the internal `SimpleConsumer` of a `BalancedConsumer`
that a rebalance is in progress, fixing a bug causing partitions to be unreleased
* Fixed a bug causing busywaiting in the `BalancedConsumer` when there are no partitions
available
* Updated the protocol implementation to send non-empty `GroupMembershipProtocol`
objects and become compatible with the Administrative API
* Fixed a bytestring bug causing `kafka_tools.reset_offsets` not to work in python 3
* Added a separate retry limit on connections to the offset manager
* Improved logging on socket errors
* Fixed a bug causing API version not to be passed on certain requests
* Handled new `MessageSet` compression scheme in API v1
* Fixed a bug in `rdkafka.SimpleConsumer` causing exceptions not to be raised from worker
threads
* Fixed a bug causing `fetch_offsets` not to raise exceptions under certain conditions
when it should

Miscellaneous
-------------

* Separated gevent tests from other builds in Travis
* Made dependency on gevent optional
* Added a convenient CLI entry point via `__main__`
* Fixed exception naming convention to align with naming in the broker
* Avoided building the `rdkafka` extension on platforms that don't support it
* Fixed a bug in test harness causing some tests not to be inherited from parent classes

.. _Compare 2.6.0: https://github.com/Parsely/pykafka/compare/2.5.0...2.6.0

Minor Version Features
----------------------

* Added support to `Broker` and `Cluster` for Kafka 0.10's Administrative API
* Changed the `MemberAssignment` protocol API to more closely match the schema defined
by Kafka
* Changed the rdkafka C module to return offset reports from produce requests

Bug Fixes
---------

* Changed components to use `six.reraise` to raise worker thread exceptions for easier
debugging
* Included message offset in messages returned from `Producer` delivery reports
* Changed protocol implementation to parse `ConsumerGroupProtocolMetadata` from
bytestrings returned from Kafka
* Added some safety checks and error handling to `Broker`, `Cluster`, `Connection`
* Removed update lock from `produce()`
* Add cleanup logic to `Producer` to avoid certain deadlock situations
* Change the name of the assignment strategy to match the standard `range` strategy
* Fix crash in rdkafka related to `broker.version.fallback`
* Fix nuisance error messages from rdkafka
* Handled `struct.error` exceptions in `Producer._send_request`

Miscellaneous
-------------

* Upgraded the version of PyPy used in automated tests
* Upgraded the version of python 3 and Kafka used in automated tests

.. _Compare 2.6.0.dev3: https://github.com/Parsely/pykafka/compare/2.6.0.dev2...2.6.0.dev3

Minor Version Features
----------------------

* Changed the rdkafka C module to return offset reports from produce requests

Bug Fixes
---------

* Added some safety checks and error handling to `Broker`, `Cluster`, `Connection`
* Removed update lock from `produce()`
* Add cleanup logic to `Producer` to avoid certain deadlock situations
* Change the name of the assignment strategy to match the standard `range` strategy
* Fix crash in rdkafka related to `broker.version.fallback`
* Fix nuisance error messages from rdkafka

Miscellaneous
-------------

* Upgraded the version of python 3 and Kafka used in automated tests