aiokafka

New features:

• Support static membership protocol, KIP-345 (issue #680, pr #941 by
  @patkivikram and @joshuaherrera)

Bugfixes:

• Fix extra dependencies (issue #952)

New features:

• Support static membership protocol, KIP-345 (issue #680, pr #941 by
  @patkivikram and @joshuaherrera)

New features:

• Include kafka-python into aiokafka's code base (issue #928 and others)
• Replace python-snappy and zstandard with cramjam (issue #930)
• PEP518 compliant pyproject.toml
• Python 3.12 support

Bugfixes:

• Fix type annotation for ConsumerRecord (pr #912 by @zschumacher)
• Improve send performance (issue #943)

Improved Documentation:

• Fix AbstractTokenProvider.token example (pr #919 by @mtomilov)

New features:

• Include kafka-python into aiokafka's code base
• Replace python-snappy and zstandard with cramjam
• PEP518 compliant pyproject.toml
• Python 3.12 support

Bugfixes:

• Fix type annotation for ConsumerRecord (pr #912 by @zschumacher)
• Improve send performance (issue #943)

Improved Documentation:

• Fix AbstractTokenProvider.token example (pr #919 by @mtomilov)

New features:

• Include kafka-python into aiokafka's code base
• Replace python-snappy and zstandard with cramjam
• PEP518 compliant pyproject.toml
• Python 3.12 support

Bugfixes:

• Fix type annotation for ConsumerRecord (pr #912 by @zschumacher)

Improved Documentation:

• Fix AbstractTokenProvider.token example (pr #919 by @mtomilov)

New features:

• Drop support for Python 3.7 due to end of life (pr #893)

Bugfixes:

• Add SASL authentication support to AIOKafkaAdminClient (issue #889,
  pr #890 by @selevit)

Improved Documentation:

• Update security_protocol argument docstring (issue #883, pr #884 by
  @gabrielmbmb)
• Remove incorrect await for AIOKafkaConsumer.highwater() (pr #858 by
  @yi-jiayu)

New features:

• Add codec for ZStandard compression (KIP-110) (pr #801)
• Add basic admin client functionality (pr #811 started by @gabriel-tincu)
• Drop support for Python 3.6, add support and pre-built packages for Python
  3.10 (pr #841)

Bugfixes:

• Fix KeyError on solitary abort marker (issue #781, pr #782 by @pikulmar)
• Fix handling unsupported compression codec (issue #795)
• Handled other SASL mechanism in logging (issue #852, pr #861 by @mangin)

Improved Documentation:

• Fix documentation on how to install optional features (issue #645)
• Improve the rendering of the documentation (pr #722 by @multani)
• Fix MyRebalancer example in docs/consumer.rst (pr #731 by @aamalev)

Bugfixes:

• Fix CancelledError handling in sender (issue #710)
• Fix exception for weakref use after object deletion (issue #755)
• Fix consumer's start() method hanging after being idle for more than
  max_poll_interval_ms (issue #764)

Improved Documentation:

• Add SASL_PLAINTEXT and SASL_SSL to valid values of security protocol
  attribute (pr #768 by @pawelrubin)

Bugfixes:

• Allow group coordinator to close when all brokers are unavailable (issue #659
  and pr #660 by @dkilgore90)
• Exclude .so from source distribution to fix usage of sdist tarball
  (issue #681 and pr #684 by ods)
• Add dataclasses backport package to dependencies for Python 3.6
  (pr #690 by @ods)
• Fix initialization without running loop (issue #689 and pr #690 by @ods)
• Fix consumer fetcher for python3.9 (pr #672 by @dutradda)
• Make sure generation and member id are correct after (re)joining group.
  (issue #727 and pr #747 by @vangheem)

Deprecation:

• Add deprecation warning when loop argument to AIOKafkaConsumer and
  AIOKafkaProducer is passed. It's scheduled for removal in 0.8.0 as a
  preparation step towards upcoming Python 3.10 (pr #699 by @ods)

Improved Documentation:

• Update docs and examples to not use deprecated practices like passing loop
  explicitly (pr #693 by @ods)
• Add docstring for Kafka header support in Producer.send() (issue #566 and
  pr #650 by @andreportela)

New features:

• Add support for Python 3.8 and 3.9. (issue #569, pr #669 and #676 by @ods)
• Drop support for Python 3.5. (pr #667 by @ods)
• Add OAUTHBEARER as a new sasl_mechanism. (issue #618 and pr #630 by @oulydna)

Bugfixes:

• Fix memory leak in kafka consumer when consumer is in idle state not consuming any message. (issue #628 and pr #629 by @iamsinghrajat)

New features:

• Add async context manager support for both Producer and Consumer. (pr #613 and #494 by @nimish)
• Upgrade to kafka-python version 2.0.0 and set it as non-strict
  parameter. (issue #590 by @yumendy and #558 by @originalgremlin)
• Make loop argument optional (issue #544)
• SCRAM-SHA-256 and SCRAM-SHA-512 support for SASL authentication (issue #571 and pr #588 by @SukiCZ)
• Added headers param to AIOKafkaProducer.send_and_wait (pr #553 by @megabotan)
• Add consumer.last_poll_timestamp(partition) which gives the ms timestamp of the last
  update of highwater and lso. (issue #523 and pr #526 by @aure-olli)
• Change all code base to async-await (pr #522)
• Minor: added PR and ISSUE templates to GitHub

Bugfixes:

• Ignore debug package generation on bdist_rpm command. (issue #599 by @gabriel-tincu)
• UnknownMemberId was raised to the user instead of retrying on auto commit. (issue #611)
• Fix issue with messages not being read after subscriptions change with group_id=None. (issue #536)
• Handle RequestTimedOutError in coordinator._do_commit_offsets() method to explicitly mark
  coordinator as dead. (issue #584 and pr #585 by @FedirAlifirenko)
• Added handling asyncio.TimeoutError on metadata request to broker and metadata update.
  (issue #576 and pr #577 by @MichalMazurek)
• Too many reqs on kafka not available (issue #496 by @lud4ik)
• Consumer.seek_to_committed now returns mapping of committed offsets (pr #531 by @ask)
• Message Accumulator: add_message being recursive eventually overflows (pr #530 by @ask)

Improved Documentation:

• Clarify auto_offset_reset usage. (pr 601 by @dargor)
• Fix spelling errors in comments and documentation using codespell (pr #567 by mauritsvdvijgh)
• Delete old benchmark file (issue #546 by @jeffwidman)
• Fix a few typos in docs (pr #573 and pr #563 by @ultrabug)
• Fix typos, spelling, grammar, etc (pr #545 and pr #547 by @jeffwidman)
• Fix typo in docs (pr #541 by @pablogamboa)
• Fix documentation for benchmark (pr #537 by @abhishekray07)
• Better logging for bad CRC (pr #529 by @ask)

Bugfixes:

• Fix ConnectionError breaking metadata sync background task (issue #517 and #512)
• Fix event_waiter reference before assignment (pr #504 by @romantolkachyov)
• Bump version of kafka-python

New features:

• Add SASL support with both SASL plain and SASL GGSAPI. Support also includes
  Broker v0.9.0, but you will need to explicitly pass api_version="0.9".
  (Big thanks to @cyrbil and @jsurloppe for working on this)
• Added support for max_poll_interval_ms and rebalance_timeout_ms settings (
  issue #67)
• Added pause/resume API for AIOKafkaConsumer. (issue #304)
• Added header support to both AIOKafkaConsumer and AIOKafkaProducer for
  brokers v0.11 and above. (issue #462)

Bugfixes:

• Made sure to not request metadata for all topics if broker version is passed
  explicitly and is 0.10 and above. (issue #440, thanks to @ulrikjohansson)
• Make sure heartbeat task will close if group is reset. (issue #372)

New features:

• Add full support for V2 format messages with a Cython extension. Those are
  used for Kafka >= 0.11.0.0
• Added support for transactional producing (issue #182)
• Added support for indempotent producing with enable_idempotence parameter
• Added support for fetch_max_bytes in AIOKafkaConsumer. This can help limit
  the amount of data transferred in a single roundtrip to the broker, which is
  essential for consumers with a large number of partitions

Bugfixes:

• Fix issue with connections not propagating serialization errors
• Fix issue with group=None resetting offsets on every metadata update
  (issue #441)
• Fix issue with messages not delivered in order when Leader changes (issue
  #228)
• Fixed version parsing of api_version parameter. Before it ignored the
  parameter

Fix issue #444 and #436 related to a memory leak in asyncio.shield()

The work here was concentrated on fixing bugs after the coordination refactors on v0.4.0. Hope it will serve you better now!

Bugfix:

• Added error propagation from coordinator to the main consumer. Before consumer
  just stopped with error logged. (issue #294)
• Fix manual partition assignment, broken in 0.4.0 (issue #394)
• Fixed RecursionError in MessageAccumulator.add_message (issue #409)
• Update kafka-python to latest 1.4.3 and added support for Python3.7
• Dropped support for Python3.3 and Python3.4

Infrastructure:

• Added Kafka 1.0.2 broker for CI test runner
• Refactored Travis CI build pipeline

• Fix issue when offset commit error reports wrong partition in log (issue #353)
• Add ResourceWarning when Producer, Consumer or Connections are not closed
  properly (issue #295)
• Fix Subscription None in GroupCoordinator._do_group_rejoin (issue #306)

Major changes:

• Full refactor of the internals of AIOKafkaConsumer. Needed to avoid several
  race conditions in code (PR #286, fixes #258, #264 and #261)
• Rewrote Records parsing protocol to allow implementation of newer protocol
  versions later
• Added C extension for Records parsing protocol, boosting the speed of
  produce/consume routines significantly
• Added an experimental batch producer API for unique cases, where user want's
  to control batching himself (by @shargan)

Minor changes:

• Add timestamp field to produced message's metadata. This is needed to find
  LOG_APPEND_TIME configured timestamps.
• Consumer.seek() and similar API's now raise proper ValueError's on
  validation failure instead of AssertionError.

Bug fixes:

• Fix connections_max_idle_ms option, as earlier it was only applied to
  bootstrap socket. (PR #299)
• Fix consumer.stop() side effect of logging an exception
  ConsumerStoppedError (issue #263)
• Problem with Producer not able to recover from broker failure (issue #267)
• Traceback containing duplicate entries due to exception sharing (PR #247
  by @Artimi)
• Concurrent record consumption rasing InvalidStateError('Exception is not set.') (PR #249 by @aerkert)
• Don't fail GroupCoordinator._on_join_prepare() if commit_offset()
  throws exception (PR #230 by @shargan)
• Send session_timeout_ms to GroupCoordinator constructor (PR #229 by @shargan)

Big thanks to:

• @shargan for Producer speed enhancements and the batch produce API
  proposal/implementation.
• @vineet-rh and other contributors for constant feedback on Consumer
  problems, leading to the refactor mentioned above.

• Added AIOKafkaProducer.flush() method. (PR #209 by @vineet-rh)
• Fixed a bug with uvloop involving float("inf") for timeout. (PR #210 by
  dmitry-moroz)
• Changed test runner to allow running tests on OSX. (PR #213 by @shargan)