spring-kafka

⭐ New Features

• Allow to use KafkaListenerContainerFactory with custom message converter for batch and record listeners #2566
• Repeating headers in Non-Blocking Retry Mechanism #2549

🐞 Bug Fixes

• ProducerFactory's argument txIdPrefix of createProducer not marked as @Nullable #2583
• Container Does Not Pause for Missing Acks After a Rebalance #2581
• Non-blocking retries: partition rebalance causes emergency consumer stop #2576
• NPE in FailedBatchProcessor if Reported Index is Out of Bounds #2558
• An application using default spring boot configuration fails at the startup when there are multiple @KafkaListener with @RetryableTopic #2554

📔 Documentation

• The documentation of the autoFlush seems to be confusing (to me) #2562
• GH-2516: Fix code example in the documentation for for Non-Blocking Retries - DLT Failure Behavior #2517

🔨 Dependency Upgrades

• Upgrade Jackson, Micrometer, Reactor, Spring Data, Spring Framework Versions #2592
• Upgrade to Apache Kafka 3.3.2 #2552

🔨 Tasks

• Update Gradle Enterprise plugin #2578

❤️ Contributors

Thank you to all the contributors who worked on this release:

@jprinet and @jucosorin

External Links

• Reference Documentation

⭐ New Features

• GH-2566: Both Converters in Container Factory #2570
• Repeating headers in Non-Blocking Retry Mechanism #2551

🐞 Bug Fixes

• Container Does Not Pause for Missing Acks After a Rebalance #2587
• ProducerFactory's argument txIdPrefix of createProducer not marked as @Nullable #2585
• Non-blocking retries: partition rebalance causes emergency consumer stop #2580
• GH-2565: Topic partition is not resumed for RetryTopic #2569
• GH-2558: Fix Possible NPE in FailedBatchProcessor #2560
• The RetryTopicBeanNames.DEFAULT_SCHEDULER_WRAPPER_BEAN_NAME accidentally points to defaultRetryTopicKafkaTemplate #2557

📔 Documentation

• Update Docs For Overriding Boot Dependencies #2564
• GH-2516: Fix code example in the documentation for for Non-Blocking Retries - DLT Failure Behavior #2556

🔨 Dependency Upgrades

• Upgrade Jackson, Micrometer, Reactor, Spring Data Versions #2591

❤️ Contributors

Thank you to all the contributors who worked on this release:

@cenkakin and @jucosorin

External Links

• Reference Documentation

⭐ New Features

• GH-2542: FallbackBatchErrorHandler Improvement #2543
• Add possibility to remove custom headers in method: addHeadersFunction(...) from DeadLetterPublishingRecoverer #2528
• Cluster Id: Use Producer/Consumer Bootstrap Servers if Different to the Context's KafkaAdmin Servers #2505

🐞 Bug Fixes

• Get Cluster Id for Observation Lazily #2538
• Topic partition is not resumed after re-assignment for RetryTopic #2525
• KafkaListener Regression - Pseudo Bean __listener Not Available in the Id Attribute #2521

📔 Documentation

• Remove outdated information for transactional.id #2524
• GH-2511: Fix code examples in the documentation for for Non-Blocking Retries - BackOff Configuration #2512

🔨 Dependency Upgrades

• Upgrade Spring, Micrometer, Reactor Versions #2548

🔨 Tasks

• kafka-clients 3.3.2 Compatibility #2540
• Refactor Switch Expressions #2535
• Upgrade Samples to 3.0.x #2508

❤️ Contributors

Thank you to all the contributors who worked on this release:

@LukasHeimann, @jucosorin, and @truepele

External Links

• Reference Documentation

⭐ New Features

• GH-2542: FallbackBatchErrorHandler Improvement #2544
• Add possibility to remove custom headers in method: addHeadersFunction(...) from DeadLetterPublishingRecoverer #2530

🐞 Bug Fixes

• Topic partition is not resumed after re-assignment for RetryTopic #2534

📔 Documentation

• GH-2511: Fix code examples in the documentation for for Non-Blocking Retries - BackOff Configuration #2513

🔨 Dependency Upgrades

• Upgrade Spring, Micrometer, Reactor Versions #2547

🔨 Tasks

• kafka-clients 3.3.2 Compatibility #2541

❤️ Contributors

Thank you to all the contributors who worked on this release:

@jucosorin and @truepele

External Links

• Reference Documentation

⭐ New Features

• Add Option to Set Listener Thread Name #2501

🐞 Bug Fixes

• Error on executing tests with spring-kafka-test 3.0.0 #2490
• Duplicate consumption triggered by consumer group rebalance #2489

📔 Documentation

• Document Parameter Annotation Limitations #2499

🔨 Dependency Upgrades

• Upgrade Spring Framework (6.0.3), Micrometer (1.10.2), Reactor (2022.0.1) #2507

❤️ Contributors

Thank you to all the contributors who worked on this release:

@mikael-carlstedt

🐞 Bug Fixes

• Duplicate consumption triggered by consumer group rebalance #2492

📔 Documentation

• Fix Consumer Thread Naming Documentation #2504
• Document Parameter Annotation Limitations #2500

❤️ Contributors

Thank you to all the contributors who worked on this release:

@mikael-carlstedt

⭐ New Features

• Make RetryTopicConfigurationSupport initialize RetryTopicComponentFactory lazily #2478
• Use isEmpty() Instead of size(); toList() #2474
• Add spring-kafka-bom #2472
• suggestion: change @RetryableTopic replicationFactor default to -1 (broker default) #2471

🐞 Bug Fixes

• Disable Global Embedded Kafka Support if JUnit Platform is < 1.8 #2481
• java.lang.IllegalStateException: Only one 'RetryTopicConfigurationSupport' is allowed when Spring context is reloaded #2477

📔 Documentation

• Docs for Native Images #2483

🔨 Dependency Upgrades

• Spring Data 2022.0.0, Retry 2.0.0, Framework 6.0.0 #2488
• Hibernate Validation 8.0.0-final, Micrometer 1.10.1, Mockito 4.8.1, Reactor 2022.0.0 #2487
• Micrometer 1.10.0, Tracing, Docs 1.0.0 #2486
• Upgrade to Jackson 2.14.0 #2485

Change log:

5a412d810 Upgrade Versions; Prepare for Release
91ac6d217 GH-2477: Fix Static State RetryTopicConfigSupport
8c15f7d0d Fix Missing Re-Interrupt
42f654d2c Revert KafkaAdmin Change (Previous Commit)
30d2b8983 Fix Missing Re-Interrupts
65bbdb666 Add KafkaTestUtils methods using Duration for timeout (2.9) (#2468)
c92ec760c GH-2464: Fix Delivery Attempt Header (Rare)
921e6ae5c GH-2459: Resolve Sonar Issues
f9f6a559e @RetryableTopic More SpEL Support
5c70ec182 GH-2459: FallbackBatchErrorHandler Retryable Ex
2966c7d05 GH-2451: Fix Class Level Listener Multi Instances
9e0703557 GH-2352: Fix Doc Typo
2bf7d6244 GH-2352: Expose KLERegistrar Getter on BPP

Change log:

e31e639f6 Upgrade Versions; Prepare for Release
5df71c169 Fix Missing Re-Interrupt
a9b0252be Revert KafkaAdmin Change (Previous Commit)
8c2b32560 Fix Missing Re-Interrupts
6ec1b3bb0 GH-2459: FallbackBatchErrorHandler Retryable Ex
48b2017b0 GH-2451: Fix Class Level Listener Multi Instances

Change log:

b8bccd914 Revert Micrometer Docs to RC1
c51d6259d Upgrade to Spring Framework 6.0.0-RC3
0d78fbe53 Fix Get ObservationRegistry
8217a9e81 Fix Missing Re-Interrupts
4f206733d Address Sonar Issues
df9b014b3 NonNull API Retryable Topic
9d43b9313 Address Sonar Issues
44db5096c Add KafkaTestUtils methods using Duration for timeout (#2467)
59eb2dcca GH-2464: Fix Delivery Attempt Header (Rare)
a00789a69 GH-2441: Merged Annotations with RetryableTopic
954b62254 GH-2459: Resolve Sonar Issues
72d428478 @RetryableTopic Support SpEL for listenerContainerFactory and kafkaTemplate properties
e38c4cbb7 GH-2282: Fix Multi RetryableTopic Same Topic
70b9143c9 GH-2459: FallbackBatchErrorHandler Retryable Ex
97ed09ede GH-2427: Allow RuntimeException to be Classified
4fedd755c GH-2443: Distinct Concurrency for Retry Containers
62300bb33 GH-2451: Fix Class Level Listener Multi Instances
8ccd8639d GH-2198: Observability Doc Gen Polishing
c6bbba8b5 Update the usage of micrometer-docs-generator
f1ca2fd3c GH-2448: State Non-Blocking Retries API Status
ada628abc GH-2352: Fix Doc Typo
b743f035e GH-2352: Expose KLERegistrar Getter on BPP

Change log:

a9c8a827 Upgrade Versions; Prepare for Release
1400fbd7 GH-2438: RetryTopic Destination Partition Select
67e0ed12 Add meta-annotation support to RetryableTopic (#2440)
a9cebb0c GH-2432: Remove Unnecessary Variable
86f6448a GH-2432: Fix Retryable Topic Provisioning
6536f3c6 GH-2410: Disallow nack() with Out of Order Commits
1edbbac7 GH-2430: Fix Unnecessary describeCluster()
7290bc16 GH-2430: Fix TimeUnit in describeCluster Calls
4bcb503a Update config example type in doc
d4619f03 GH-2198: Observability Documentation
239e3506 GH-2423: Upgrade to Kafka 3.3.1, Streams Proc. API
f4084903 Fix Race In Test
7cc45245 GH-2419: DLPR: Protect Against Non-Compliant PF
b0c1c98d GH-2387: Fix FallbackBatchErrorHandler Events
f079f615 Don't Use kafka.common.KafkaException in Broker
bb34b9cd GH-2415: Doc for DeserializationEx... with DLPR
8a23e479 GH-2348: Custom Correlation Consumer Side
90665980 Add Properties to Deprecation Annotations
0da801f7 GH-2404: Refactor FallbackBatchErrorHandler
884a43d1 GH-2198: Fix Race in ObservationTests
e531a0f8 GH-2395: RetryListener - Add Batch Methods
deaded0e Upgrade Apache Kafka to 3.2.3
8b4b601a GH-2399: ReplyingKT Human Readable Correlation
10b75a32 GH-2400: ReplyingKT Improve CorrelationId Logging
5fb9b8e6 GH-2198: Remove Unused Field from Test

Change log:

6b938e4f Upgrade Versions; Prepare for Release
e4554316 GH-2438: RetryTopic Destination Partition Select
58aa8b18 GH-2432: Remove Unnecessary Variable
9850af5d GH-2432: Fix Retryable Topic Provisioning
cb5c36b1 GH-2410: Disallow nack() with Out of Order Commits
91507a91 Update config example type in doc
c69791e8 Add Acknowledgment.nack() variants accepting Duration
43bc67a2 Fix Race In Test
3850a012 GH-2419: DLPR: Protect Against Non-Compliant PF
5c776cd5 GH-2387: Fix FallbackBatchErrorHandler Events
7c648d76 Don't Use kafka.common.KafkaException in Broker
766333c1 GH-2415: Doc for DeserializationEx... with DLPR
8ad588ee GH-2395: RetryListener - Add Batch Methods
f584d8f2 Upgrade Apache Kafka to 3.2.3
5fed9035 GH-2400: ReplyingKT Improve CorrelationId Logging

Change log:

e021e32d Upgrade Versions; Prepare for Release
bd146f62 GH-2432: Remove Unnecessary Variable
6a253f23 GH-2432: Fix Retryable Topic Provisioning
950a2ea7 GH-2410: Disallow nack() with Out of Order Commits
0547dc77 Update config example type in doc
a03f67bc Fix Race In Test
5bd08c08 GH-2419: DLPR: Protect Against Non-Compliant PF
f6cbf116 GH-2387: Fix FallbackBatchErrorHandler Events
2e1feb3f Don't Use kafka.common.KafkaException in Broker
34c48f5e GH-2415: Doc for DeserializationEx... with DLPR
5df2568d Fix previous commit conflicts
8cfd1d8b GH-2395: RetryListener - Add Batch Methods

Change log:

87191e09 Upgrade Versions; Prepare for Milestone Release
182c580d GH-2198: Spring Observability Initial Commit (#2394)
1b9716d5 Update Gradle Enterprise plugin to 3.11.1
f381a1cc Upgrade Apache Kafka to 3.2.2
7d19d0de Refactor - use method referencing in lambdas where possible
e5d23340 GH-2184: Don't Log KafkaBackoffException
f1253d47 GH-2387: Add Missing Pause/Resume Events
950f8cad Remove LisenableFuture Callback Extension
eb7893cc Remove RHU.registerSynthesizedAnnotation()
b6e65022 GH-2382: Fix Unchecked Cast in Test
95663255 Move to Micrometer Snapshots
b7a2ab93 Remove Another Dead Link from Change History
93004c5a Remove Dead Links from Change History
20f75bc3 Fix Duplicate Doc Anchors
afb2b529 Use spring-asciidoctor-backends and remove spring-doc-resources, to pick up the new look and feel and features.
5fc34906 Fix typo in changes-since-1.0.adoc
612eb70c Remove sharing of the retrying field in FallbackBatchErrorHandler (#2383)
b42a1259 GH-2372: Fix Race in New Test
51db0b6e GH-2372: Commit Offsets for Skipped Records
083d0d84 AOT RuntimeHints Polishing
f360f8d7 GH-2375: Add KLER.unregisterListenerContainer
3d8dcce5 GH-2359: Initial AOT Support for Native
eb7648af GH-2357: Remove Remaining Uses of ListenableFuture
f4b73afd Fix NPE in Global Embedded Kafka
ed6673ef GH-2357: Switch to CompletableFuture
a9ec9197 GH-2355: Add ManualAckListenerErrorHandler (#2356)

Change log:

fabdcb73 Upgrade Versions; Prepare for Release
5efa3357 Upgrade Apache Kafka to 3.2.2
4fdedd7f GH-2184: Don't Log KafkaBackoffException
83364615 Deprecate ListenableFuture Callback Extensions
ea6fa8fa Add Missing Pause/Resume Events
781e7261 GH-2382: Fix Unchecked Cast in Test
2eacf7ad Remove Another Dead Link from Change History
8f6fd87f Remove Dead Links from Change History
7da485a0 Fix Duplicate Doc Anchors
f9419c23 Fix typo in whats-new.adoc
6a28cc5e GH-2382: Fix FBEH Cross Talk
798dea3f GH-2375: Add KLER.unregisterListenerContainer
90c6e957 GH-2357: Remove Remaining Uses of ListenableFuture

Change log:

48334e4b Upgrade Reactor Version
269fd79a Upgrade Versions; Prepare for Release
8193cd2a Upgrade Apache Kafka to 3.0.2
a190ba77 Add Missing Pause/Resume Events
e1bb1945 GH-2382: Fix Unchecked Cast in Test
e4eedb70 Remove Another Dead Link from Change History
6d728168 Remove Dead Links from Change History
f3837a14 Fix Duplicate Doc Anchors
3de1e89b GH-2382: Fix RetryingBatchErrorHandler Cross Talk
f001423d GH-2375: Add KLER.unregisterListenerContainer
b6b4c1ae GH-2363: Fix Class Cast in ErrorHandlerAdapter

Change log:

7a3223d9 Upgrade Versions; Prepare for Release
295b68cb GH-2357: Fix Sonar Issues
128372f0 GH-2357: Switch to CompletableFuture
a4ee0666 GH-2355: Add ManualAckListenerErrorHandler (#2356)
e682c878 GH-2350: Fix Paused Partitions After Rebalance
2732303e GH-2344: AggReplyingKT Support Custom Correlation
b1d97b92 GH-2345: Fix Possible NPE in KafkaAdmin
4e6d59c7 Sonar Fixes
e579c620 Fix for Java 8 compatibility
dd981e1a GH-2239: Fix Boot AutoConfiguration
15e56685 GH-2340: Fix Retrying Batch Error Handling
e220b17c Sonar Fixes
11e03c91 GH-2262: Fix Possible NPE
f7cbbfeb Sonar Fixes

Change log:

4924f4c4 Fix Tests for Latest Spring Milestone
b9f7c73c Upgrade Kotlin to 1.7; Fix Micro. Tracing Jar
5fbf6835 Uprade Versions; Prepare for Milestone Release
4aa5c2af GH-2350: Fix Paused Partitions After Rebalance
6d4cba53 Remove provided deps from the target POM
58b6889c GH-2344: AggReplyingKT Support Custom Correlation
c978b866 GH-2345: Fix Possible NPE in KafkaAdmin
1026edd5 Sonar Fixes
2f500889 GH-2239: Fix Boot AutoConfiguration
edfbd1de GH-2340: Fix Retrying Batch Error Handling
9e123ef8 Sonar Fixes
3db012f7 Sonar Fixes
ce5a8add More What's New
f9e63c2f Update What's New
81c9e604 Fix JavaDocs errors
6a24685b GH-2288: Delegating EH - Traverse Causes
37910b60 GH-2239: RetryableTopic Refactoring
a63359c1 GH-2332: Fix Partitions in Pause Event
a6cce8af GH-2335: Fix RetryTopic Cause Header Name
c0e210c6 GH-2332: Fix Container.pause() with Manual Assign.
d7b100b4 Fix Appendix (Kafka Version Override)
bb26e058 GH-2280: Add ContainerProperties.pauseImmediate
ad484913 GH-2195: Fix Remaining ConsumerRecords and Test
b2b41d94 GH-2195: Fix Fix No Seek Retries with Pause
1eca6737 GH-2329: Fix interceptBeforeTx for Non-Kafka TM
36665e9d GH-2195: Polishing
e9b5e69a GH-2195: Fix No Seek Retries with Non-Blocking
20cbc759 Fix Quick Tour Clients Version
aca14efb Polish Container setTransactionManager Javadocs
2fcb082f GH-2321: Support Inbound Header Mapping Matchers
2c5c88ce GH-2622: Fix Possible NPE
343f5589 GH-2262: Docs
17f8f85d GH-2262: Polishing
7c52853c GH-2262: Add BackOffHander
6fbc6cb0 GH-2284: Publish Auth Error Events
f3215c8d GH-2318: ReplyingKT - Wait for Assignment
85061a4d Fix Previous Commit
9b97f461 Fix Possible Race in KafkaAdmin
d6fb1723 Increase PPBOManagerFactory Test Coverage (Sonar)
0f695537 GH-666: Single global embedded Kafka with JUnit 5 (#2308)
596f7d94 Widen Delegate Type for KafkaBackoffAwareMLAdapter
b57c608f Use KafkaListenerEndpoint instead of MethodKafkaListenerEndpoint in the RetryTopicNamesProvider
a372cc4a Update Gradle Enterprise plugin
70a7274c Fix Sonar Issues
12c35403 Fix Sonar Issues
db617ec6 GH-2195: Fix No Seek Retry Mode (#2267)
8ea39c4f GH-2306: Improve DeserEx Message For Improper Ex.
a07ace92 Add Converter Factories to Method Factory
a5a28cc3 GH-2304: Fix SendTo on Interface Etc.
f6c0996c GH-2297: Add KLERegistry.alwaysStartAfterRefresh
a6aff279 GH-1482: Make Topic Config Update Optional
3861e441 GH-1482: Support Admin.incrementalAlterConfigs
ef48cfa1 Add Acknowledgment.nack() variants accepting Duration (#2281)
4d4a1a68 GH-2295: No Resolvers with ConsumerRecordMetadata
2112c8dd Log Uncommitted After Rebalance
0c65c9be GH-2292: Doc - Fix Container Factory Generics
8034c2cf GH-2068: Add lightweight converting adapter (#2165)
c9b74e17 GH-2286: Doc Fixes
93f5ae5c Precise unit for sleep duration and wake time
ed0a4924 GH-2148: Use ObjectProvider to Locate MeterRegistry

Change log:

16fbdde6 Upgrade Versions; Prepare for Release
e94985b4 GH-2350: Fix Paused Partitions After Rebalance
f1f200cc GH-2344: AggReplyingKT Support Custom Correlation
f747329c GH-2345: Fix Possible NPE in KafkaAdmin
dee71ae7 GH-2340: Fix Retrying Batch Error Handling
3f306732 GH-2288: Delegating EH - Traverse Causes
4b93cdfb KafkaAdmin: fix usage of the Map.of()
989fb298 GH-2332: Fix Container.pause() with Manual Assign.
b2862506 GH-2335: Fix RetryTopic Cause Header Name
3eb5d9fe Fix Doc Copyright
fea092cf transactionIdPrefix Doc Polishing
bdb7e844 Polish Container setTransactionManager Javadocs
46c63e2f GH-2321: Support Inbound Header Mapping Matchers
558c19fd GH-2318: ReplyingKT - Wait for Assignment
e49d8506 Fix Possible Race in KafkaAdmin
d1b60d27 Widen Delegate Type for KafkaBackoffAwareMLAdapter

Change log:

1013095c Fix JavaDocs errors
948db6eb GH-2288: Delegating EH - Traverse Causes
ed0f2e5f Update What's new For RC1
b88cbf90 GH-2239: RetryableTopic Refactoring
131b0e7a GH-2332: Fix Partitions in Pause Event
a6daf04c GH-2335: Fix RetryTopic Cause Header Name
1f5ae615 GH-2332: Fix Container.pause() with Manual Assign.
a3d1f42e Fix Appendix (Kafka Version Override)
55ff86a1 Fix ConcMLCMockTests for current RecInterceptor
19a9a457 GH-2280: Add ContainerProperties.pauseImmediate
6f448465 GH-2195: Fix Remaining ConsumerRecords and Test
056cfe3c GH-2195: Fix Fix No Seek Retries with Pause
21172def GH-2329: Fix interceptBeforeTx for Non-Kafka TM
72f23e74 GH-2195: Polishing
16c3f2f3 Some fixes for current code base state
a24a2633 GH-2195: Fix No Seek Retries with Non-Blocking
1b344056 Fix Doc Copyright
d7f4f4b8 Fix Quick Tour Clients Version
89c39815 transactionIdPrefix Doc Polishing
bbd1887d Polish Container setTransactionManager Javadocs
ff228811 GH-2321: Support Inbound Header Mapping Matchers
74bfafa0 GH-2262: Fix Doc Anchor
6b3bda9a GH-2262: Docs
18b8322a GH-2262: Polishing
2ba047a0 GH-2262: Add BackOffHander
151f83d2 GH-2318: ReplyingKT - Wait for Assignment
80ceca81 Fix Possible Race in KafkaAdmin
6fad9987 Increase PPBOManagerFactory Test Coverage (Sonar)
5fd9c3d3 Widen Delegate Type for KafkaBackoffAwareMLAdapter
78fa932f RetryTopicNamesProvider: Refactor Endpoint Methods
a4906aa6 Update Gradle Enterprise plugin
a2774fb7 Update Gradle Enterprise plugin
f8bd0b00 Upgrade Versions; Prepare for Release
bb609760 Fix Sonar Issues
3c425a14 Fix Sonar Issues
7cb0ff80 Fix compatibility with current Kafka Client
2f622055 GH-2195: Fix No Seek Retry Mode (#2267)
02132686 GH-2306: Improve DeserEx Message For Improper Ex.
960d6a4a Add Converter Factories to Method Factory
62ecbf05 GH-2304: Fix SendTo on Interface Etc.
93ee7ac7 GH-2297: Add KLERegistry.alwaysStartAfterRefresh
be7fdb46 GH-1482: Make Topic Config Update Optional
5410d152 GH-1482: Support Admin.incrementalAlterConfigs
460098a4 GH-2295: No Resolvers with ConsumerRecordMetadata
8f7f55bc Log Uncommitted After Rebalance
daf72332 GH-2292: Doc - Fix Container Factory Generics
9248d830 GH-2286: Doc Fixes
ea3dd767 Precise unit for sleep duration and wake time
38321247 GH-2148: Use ObjectProvider to Locate MeterRegistry
458ebad6 GH-2274: Option to Not Configure (De)serializers
28421968 Upgrade kafka-clients to 3.2.0; other upgrades
6473c333 GH-2269: Improve DLPR Extensibility
968df682 GH-2252: Keep offset metadata in case of batch reprocessing (#2253)
c107857f GH-2128: More Doc Polishing
80243edc GH-2260: Reduce Header Constant Verbosity
4a5f4a18 GH-2249: Batch Listener LISTENER_INFO Headers
ab9d645e GH-2225: Fix New Package Tangle
76de24db GH-2128: Document Nack Sleep Time Limitations