ruby-kafka

What's Changed

• Instrument create_time from last message in batch by @jonmagic in https://github.com/zendesk/ruby-kafka/pull/811
• Add client function for fetching topic replica count by @lairen in https://github.com/zendesk/ruby-kafka/pull/822
• [Feat] Allow consumers to refresh the topic lists by @lmduc in https://github.com/zendesk/ruby-kafka/pull/818
• Disconnect after leaving the group by @abicky in https://github.com/zendesk/ruby-kafka/pull/817
• Issue 823 > Use max_wait_time while sleeping in fetch_batches by @ElenaHenderson in https://github.com/zendesk/ruby-kafka/pull/825
• Add kafka 2.5 to CI by @geoff2k in https://github.com/zendesk/ruby-kafka/pull/830
• Make verify_hostname settable for ssl contexts by @lairen in https://github.com/zendesk/ruby-kafka/pull/828
• Support Unix domain socket in Kafka::Datadog by @abicky in https://github.com/zendesk/ruby-kafka/pull/827
• Critical bug fix: Extra sanity checking when marking offsets as processed by @theturtle32 in https://github.com/zendesk/ruby-kafka/pull/824
• Use the standard SPDX ID for license in gemspec by @kenhys in https://github.com/zendesk/ruby-kafka/pull/832
• Add some hardening by @dasch in https://github.com/zendesk/ruby-kafka/pull/833

New Contributors

• @lairen made their first contribution in https://github.com/zendesk/ruby-kafka/pull/822
• @lmduc made their first contribution in https://github.com/zendesk/ruby-kafka/pull/818
• @ElenaHenderson made their first contribution in https://github.com/zendesk/ruby-kafka/pull/825
• @geoff2k made their first contribution in https://github.com/zendesk/ruby-kafka/pull/830
• @kenhys made their first contribution in https://github.com/zendesk/ruby-kafka/pull/832

Full Changelog: https://github.com/zendesk/ruby-kafka/compare/v1.0.0...v1.1.0

What's Changed

• Various AsyncProducer improvements by @cstyles in https://github.com/zendesk/ruby-kafka/pull/855
• Log RebalanceInProgress in ConsumerGroup#heartbeat as warning by @marcoroth in https://github.com/zendesk/ruby-kafka/pull/858
• change parameter to ssl_verify_hostname in readme by @contrun in https://github.com/zendesk/ruby-kafka/pull/856
• Support custom assignment strategy by @abicky in https://github.com/zendesk/ruby-kafka/pull/846
• Positional and keyword argument handling in Ruby 2.7 by @tsrivishnu in https://github.com/zendesk/ruby-kafka/pull/859
• Improved Exceptions in TransactionManager by @jwilger in https://github.com/zendesk/ruby-kafka/pull/862

New Contributors

• @cstyles made their first contribution in https://github.com/zendesk/ruby-kafka/pull/855
• @marcoroth made their first contribution in https://github.com/zendesk/ruby-kafka/pull/858
• @contrun made their first contribution in https://github.com/zendesk/ruby-kafka/pull/856
• @tsrivishnu made their first contribution in https://github.com/zendesk/ruby-kafka/pull/859

Full Changelog: https://github.com/zendesk/ruby-kafka/compare/v1.2.0...v1.3.0

What's Changed

• fix Kafka::TransactionManager#send_offsets_to_txn by @stasCF in https://github.com/zendesk/ruby-kafka/pull/866
• Add kafka 2.6.0 to Circle CI by @methodmissing in https://github.com/zendesk/ruby-kafka/pull/869
• Resolve "Passing the keyword argument as ..." deprecation warning by @abicky in https://github.com/zendesk/ruby-kafka/pull/875
• Resolve RSpec::Mocks::OutsideOfExampleError by @abicky in https://github.com/zendesk/ruby-kafka/pull/874
• Make "consuming messages with a custom assignment strategy" stable by @abicky in https://github.com/zendesk/ruby-kafka/pull/876
• kafka 2.7.0 by @vvuibert in https://github.com/zendesk/ruby-kafka/pull/880
• Add murmur2_random support for message partitioning. by @divo in https://github.com/zendesk/ruby-kafka/pull/884
• Support seed brokers' hostname with multiple addresses by @abicky in https://github.com/zendesk/ruby-kafka/pull/877
• Fix "@param tag has unknown parameter name" by @abicky in https://github.com/zendesk/ruby-kafka/pull/897
• Updated to ignore all control batches by @rkruze in https://github.com/zendesk/ruby-kafka/pull/893
• Handle SyncGroup responses with a non-zero error and no assignments by @BrentWheeldon in https://github.com/zendesk/ruby-kafka/pull/896
• Refresh metadata if necessary on deliver_message by @wamaral in https://github.com/zendesk/ruby-kafka/pull/901
• ISSUE-525 / 764: ruby-kafka does not support different topic subscriptions in the same consumer group by @eduardopoleoflipp in https://github.com/zendesk/ruby-kafka/pull/903
• Make ssl_ca_cert_file_path support an array of files by @syedriko in https://github.com/zendesk/ruby-kafka/pull/905
• This spec stopped working at some point by @dasch in https://github.com/zendesk/ruby-kafka/pull/909
• Fix multiple [Producer name] tags on failures by @ojab in https://github.com/zendesk/ruby-kafka/pull/910

New Contributors

• @stasCF made their first contribution in https://github.com/zendesk/ruby-kafka/pull/866
• @methodmissing made their first contribution in https://github.com/zendesk/ruby-kafka/pull/869
• @divo made their first contribution in https://github.com/zendesk/ruby-kafka/pull/884
• @rkruze made their first contribution in https://github.com/zendesk/ruby-kafka/pull/893
• @BrentWheeldon made their first contribution in https://github.com/zendesk/ruby-kafka/pull/896
• @wamaral made their first contribution in https://github.com/zendesk/ruby-kafka/pull/901
• @eduardopoleoflipp made their first contribution in https://github.com/zendesk/ruby-kafka/pull/903
• @syedriko made their first contribution in https://github.com/zendesk/ruby-kafka/pull/905
• @ojab made their first contribution in https://github.com/zendesk/ruby-kafka/pull/910

Full Changelog: https://github.com/zendesk/ruby-kafka/compare/v1.3.0...v1.4.0

• Add support for config entries in the topic creation API (#540).
• Don't fail on retry when the cluster is secured (#545).

• Add support for the topic deletion API (#528).
• Add support for the partition creation API (#533).
• Allow passing in the seed brokers in a positional argument (#538).

• Instrument the start of message/batch processing (#496).
• Mark Client#fetch_messages as stable.
• Fix the list topics API (#508).
• Add support for LZ4 compression (#499).
• Refactor compression codec lookup (#509).
• Fix compressed message set offset bug (#506).
• Test against multiple versions of Kafka.
• Fix double-processing of messages after a consumer exception (#518).
• Track consumer offsets in Datadog.

• Restart the async producer thread automatically after errors.
• Include the offset lag in batch consumer metrics (Statsd).
• Make the default max_wait_time more sane.
• Fix issue with cached default offset lookups (#431).
• Upgrade to Datadog client version 3.

• Drops support for Kafka 0.9 in favor of Kafka 0.10 (#381)!
• Handle cases where there are no partitions to fetch from by sleeping a bit (#439).
• Handle problems with the broker cache (#440).
• Shut down more quickly (#438).

Requires Kafka 0.10.1+ due to usage of a few new APIs.

• Fix bug when using compression (#458).
• Update the v3 of the Fetch API, allowing a per-request max_bytes setting (#468).
• Make #deliver_message more resilient using retries and backoff.
• Add support for SASL SCRAM authentication (#465).
• Refactor and simplify SASL code.
• Fix issue when a consumer resets a partition to its default offset.
• Allow specifying a create time for messages (#481).