Ruby and Rails efficient Kafka processing framework
OTHER License
Bot releases are visible (Hide)
Published by mensfeld about 1 year ago
#attempt
method on a consumer to directly indicate number of attempt of processing given data.Published by mensfeld about 1 year ago
Karafka::Admin#read_watermark_offsets
fetching upon not_leader_for_partition
that can occur mostly on newly created topics in KRaft and after crashes during leader selection.Published by mensfeld about 1 year ago
#nil?
instead of #present?
on DataDog::Tracing::SpanOperation
(vitellochris)#subscription_group
reference consistent in the Routing and Instrumentation.subscription_group
visibility to the consumer.Published by mensfeld about 1 year ago
Published by mensfeld about 1 year ago
Published by mensfeld about 1 year ago
karafka-core
>=
2.2.2
Published by mensfeld about 1 year ago
2.2.x
karafka-core
.Published by mensfeld about 1 year ago
Karafka::Admin
setup reconfiguration via config.admin
scope.Karafka::Admin
obeys the ConsumerMapper
setup.Published by mensfeld about 1 year ago
karafka-rdkafka
.service_name
in DataDog integration.Published by mensfeld about 1 year ago
Published by mensfeld about 1 year ago
Published by mensfeld about 1 year ago
connection.client.rebalance_callback
event for instrumentation of rebalances.allow.auto.create.topics
for admin by default to prevent accidental topics creation on topics metadata lookups.query_watermark_offsets
operations by increasing too low timeout.TplBuilder
timeouts to compensate for remote clusters.connection.client.poll.error
error type reference.Published by mensfeld about 1 year ago
karafka.rb
template.waterdrop
>= 2.6.6
due to extra LoggerListener
API.Published by mensfeld about 1 year ago
Karafka::BaseConsumer#used?
method to indicate, that at least one invocation of #consume
took or will take place. This can be used as a replacement to the non-direct messages.count
check for shutdown and revocation to ensure, that the consumption took place or is taking place (in case of running LRJ).messages#to_a
return copy of the underlying array to prevent scenarios, where the mutation impacts offset management.cooperative-sticky
rebalance crash issue.consumer_persistence
per subscribed topic. This is mostly useful for plugins and extensions developers.Published by mensfeld about 1 year ago
max_wait_time
option to the Iterator.Admin#read_topic
would wait for poll interval on non-existing messages instead of early exit.connection.client.rebalance_callback.error
to the LoggerListener
instrumentation hook.Published by mensfeld over 1 year ago
#read_topic
#seek
.Karafka::Errors::TopicNotFoundError
when trying to iterate over non-existing topic.waterdrop
>= 2.6.2
karafka-core
>= 2.1.1
InvalidRealOffsetUsage
to InvalidRealOffsetUsageError
to align with naming of other errors.#seek
would overwrite manual seek of a user when running LRJ.#seek
and #pause
operations take precedence over system actions.#pause
and #resume
with one underlying connection do not race-condition.Published by mensfeld over 1 year ago
#revoked?
response quality by checking the real time assignment lost state on librdkafka.#commit_offsets
and #commit_offsets!
methods in the consumer to provide ability to commit offsets directly to Kafka without having to mark new messages as consumed.librdkafka
has fixed the crashes there.#revoked?
karafka-core
>= 2.1.0
waterdrop
>= 2.6.1
Published by mensfeld over 1 year ago
#revoked?
response quality by checking the real time assignment lost state on librdkafka.#commit_offsets
and #commit_offsets!
methods in the consumer to provide ability to commit offsets directly to Kafka without having to mark new messages as consumed.librdkafka
has fixed the crashes there.karafka-core
>= 2.1.0
waterdrop
>= 2.6.1
Published by mensfeld over 1 year ago
processing_lag
and consumption_lag
on empty batch fail on shutdown usage (#1475)Published by mensfeld over 1 year ago
_spec.rb
.#retrying?
helper result value (Aerdayne).mark_as_consumed!
raising an error instead of false
on unknown_member_id
(#1461).