redpanda

Features

• You can create namespaces in Redpanda cloud using rpk cloud namespace. by @r-vasquez in [#16777](https://github.com/redpanda- data/redpanda/pull/16777)
• #16570 [#16572](https://github.com/redpanda-data/redpanda/issues/ 16572) Publish log (i.e. stderr/stdout) output from data transforms exclusively to an internally managed Redpanda topic (_redpanda.transform_logs). Data transform logs will no longer appear in broker logs. by @oleiman in [#16663](https://github. com/redpanda-data/redpanda/pull/16663)
• #16895 Add Prometheus metrics for data transforms logging by @ oleiman in #16913

Bug Fixes

• Fixes a plausible correctness issue with idempotent requests during replication failures. by @bharathv in [#16749](https:// github.com/redpanda-data/redpanda/pull/16749)
• #16129 Fixes a bug in SASL user deletion and update where usernames with a + symbol in the username were prevented from being deleted by @pgellert in [#16811](https://github.com/redpanda-data/ redpanda/pull/16811)
• #16659 Fixes a bug in the tiered storage time-based query implementation that could result in a consumer hang when consuming very old data. by @andrwng in [#16660](https://github.com/ redpanda-data/redpanda/pull/16660)
• #16717 Fixed a few oversized allocations for some admin server endpoints. by @rockwotj in #16719
• #16884 Fixed deleting Data Transforms with names that had URL unsafe characters by @rockwotj in #16885
• #16937 Fixes a bug in windowed compaction that could cause Redpanda to crash when an error occurs while reading batches. by @andrwng in [#16940](https://github.com/redpanda-data/redpanda/pull/ 16940)

Improvements

• Adds observability into producer evictions in each shard. by @bharathv in [#16839](https://github.com/redpanda-data/redpanda/ pull/16839)
• Fix large wasm module deployments by @rockwotj in #16767
• Increase data_transforms_logging_buffer_capacity_bytes from 100KiB to 500KiB by @oleiman in [#16977](https://github.com/ redpanda-data/redpanda/pull/16977)
• Large allocations are now logged by default (similar to reactor stalls) by @StephanDollberg in [#16844](https://github.com/ redpanda-data/redpanda/pull/16844)
• #16795 Added ability to change transactional manage topic properties by @mmaslankaprv in #16968
• #16831 get_cluster_uuid returns a correctly formatted string by @ andijcr in #16832
• #16888 Data Transform builds in rpk now uses tinygo v0.31.1 by @ rockwotj in #16889
• #16947 better control of memory usage in storage layer. by @ mmaslankaprv in #16963
• #16997 Added EHOSTUNREACH to retry-able error code list by @ michael-redpanda in #16998
• optimized updating leadership metadata with health reports by @mmaslankaprv in [#16709](https://github.com/redpanda-data/ redpanda/pull/16709)
• preventing large allocation in partition balancer code by @mmaslankaprv in [#16939](https://github.com/redpanda-data/ redpanda/pull/16939)
• rpk: Remove 10s timeout in rpk profile create by @r-vasquez in [#16852](https://github.com/redpanda-data/redpanda/pull/ 16852)
• PR #16682 [v23.3.x] Implement async_for_each by @travisdowns
• PR #16688 [v23.3.x] Add forward iterator to async_for_each by @ travisdowns
• PR #16691 [v23.3.x] Rethrow on unknown exceptions in fetch handler by @ballard26
• PR #16784 [v23.3.x] c/leaders: trigger leadership notification when term changes by @mmaslankaprv
• PR #16801 [v23.3.x] c/topic_table_probe: use btree_map in topic table probe by @mmaslankaprv
• PR #16829 [v23.3.x] rpk: update help text of decommission-status by @ daisukebe
• PR #16891 [v23.3.x] cmake: upgrade tinygo compiler by @rockwotj
• PR #16894 [v23.3.x] cloud_storage: Improve stale_reader test by @Lazin
• PR #16897 [v23.3.x] Fixed background apply fiber race condition in raft::state_machine_manager by @mmaslankaprv
• PR #16903 [v23.3.x] cloud_storage: various non-functional changes by @andrwng
• PR #16908 Revert "[v23.3.x] rm_stm/idempotency: fix the producer lock scope" by @bharathv
• PR #16935 [v23.3.x] fix for cluster_config_test.py::test_aliasing by @andijcr
• PR #16965 [v23.3.x] Ensure fragment_vector fragments are always <= 128KiB by @ballard26

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.3.6...v23.3.7

Bug Fixes

• Fix a bug that resulted in Redpanda ignoring until the next restart config values that were reset to their defaults. by @ztlpn in #16641
• #16624 #16625 rpk tune -- output-script: Add a missing new line in the ballast file tuner when using the --output-script flag by @r-vasquez in [#16626](https://github. com/redpanda-data/redpanda/pull/16626)

Improvements

• #16624 #16625 rpk tune -- output-script: rpk now creates a file for you if the provided file does not exist. by @r-vasquez in [#16626](https://github.com/redpanda-data/ redpanda/pull/16626)
• PR #16493 [v23.2.x] Fixed large allocation in kafka::wait_for_leaders by @mmaslankaprv
• PR #16550 [v23.2.x] rpc: Add config flag to enable/disable compression for replies by @ StephanDollberg
• PR #16641 [v23.2.x] config: update bindings when properties are reset by @andijcr

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.25...v23.2.26

Bug Fixes

• Fix a bug that resulted in Redpanda ignoring until the next restart config values that were reset to their defaults. by @ztlpn in #16638
• Prevent detecting leader epoch advancement when state is not up to date by @mmaslankaprv in [#16573](https://github.com/redpanda-data/redpanda/ pull/16573)
• #16621 Avoid a large contiguous allocation when creating thousands of topics in a single CreateTopics request. by @travisdowns in #16622
• #16627 #16628 rpk tune -- output-script: Add a missing new line in the ballast file tuner when using the --output-script flag by @r-vasquez in [#16629](https://github. com/redpanda-data/redpanda/pull/16629)

Improvements

• Validate transform code at deploy time to ensure the correct SDK is used. by @rockwotj in [#16498](https://github.com/redpanda-data/redpanda/ pull/16498)
• #16627 #16628 rpk tune -- output-script: rpk now creates a file for you if the provided file does not exist. by @r-vasquez in [#16629](https://github.com/redpanda-data/ redpanda/pull/16629)
• PR #16546 [v23.3.x] rpc: Add config flag to enable/disable compression for replies by @ StephanDollberg
• PR #16565 [v23.3.x] introduce chunked_vector by @rockwotj
• PR #16569 [v23.3.x] rpc: Disable compression for internal rpc replies by @StephanDollberg

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.3.5...v23.3.6

Features

• #16411 You can print a schema now using rpk registry schema get --print-schema. by @r-vasquez in #16412
• PR #16292 [v23.3.x] "enable by default spillover manifest" testing followups by @andijcr

Bug Fixes

• Aggregates partitions in some cloud storage metrics when the aggregate_metrics cluster config is set to true. by @ballard26 in #16344
• #16350 rpk: fixed a bug where the --password flag could not be used along with the new configuration flag -X pass in clusters where basic authentication was enabled. by @r-vasquez in #16351
• #16389 Prevent oversized allocation with large amounts of controller metadata by @rockwotj in #16390
• #16391 Fixes a bug that may prevent redpanda from shutting down cleanly when auditing is enabled by @graphcareful in #16392
• #16393 Fix graceful shutdown of the TS archive area retention procedure. by @Lazin in #16394
• #16404 Fixes issue that causes the connection to hang when an unsupported compression type is passed via an incremental_alter_configs request by @graphcareful in #16406
• #16450 #16451 Fix an issue where create topics responses would show incorrect partition count and replication factor by @oleiman in #16452
• #16500 Fix assertion triggered by interleaving of log flush and log truncation followed by append by @Lazin in #16501
• #16517 Fix timequery error that triggered full partition scan by @Lazin in #16518

Improvements

• Adds a new cluster configuration property fetch_read_strategy. This property determines which fetch execution strategy Redpanda will use to fulfill a fetch request. The newly introduced non_polling execution strategy is the default for this property with the polling strategy being included to make backporting possible. by @ballard26 in #16484

• Improved handling of follower fetching offset validation when used with relaxed consistency by @mmaslankaprv in #16522

• Improves observability by allowing Redpanda to detect that some internal processes are stuck. by @Lazin in #16476

• Introduces a new non-polling fetch execution strategy that decreases CPU utilization of fetch requests and fetch request latency. by @ballard26 in #16484

• Publish total reclaimable space to avoid stuck decommission scenario. by @dotnwat in #16422

• SIMD instructions are generated by default for WebAssembly binaries when building with rpk. by @rockwotj in #16403

• • #16362 rpk cluster health: now --exit-when-healthy enables --watch when provided. by @r-vasquez in #16363

• PR #16294 [v23.3.x] transform-sdk/rust: borrow output record by @rockwotj

• PR #16300 [v23.3.x] Transform logging data model by @oleiman

• PR #16316 [v23.3.x] transform-sdk/rust: update rustdocs by @rockwotj

• PR #16345 [v23.3.x] Wrapped logging with vlog macro in places that missed it by @mmaslankaprv

• PR #16401 [v23.3.x] cloud_storage_clients/client_pool: handle broken _self_config_barrier by @andijcr

• PR #16432 [v23.3.x] Fixed large allocation in kafka::wait_for_leaders by @mmaslankaprv

• PR #16436 [v23.3.x] Introduce transform::logging::manager by @oleiman

• PR #16439 [v23.3.x] rptest: Fix s3.copy_object when running on GCP by @savex

• PR #16492 [v23.3.x] k/metadata: guesstimate leader when information is not yet present by @mmaslankaprv

• PR #16528 Use the original fetch impl by default in backports by @ballard26

• PR #16531 [v23.3.x] Increase default value of rpc_client_connections_per_peer to 32 by @ballard26

• Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.3.4...v23.3.5

Bug Fixes

• #16348 rpk: fixed a bug where the --password flag could not be used along with the new configuration flag -X pass in clusters where basic authentication was enabled. by @r-vasquez in #16349
• #16387 Prevent oversized allocation with large amounts of controller metadata by @rockwotj in #16388
• #16405 Fixes issue that causes the connection to hang when an unsupported compression type is passed via an incremental_alter_configs request by @graphcareful in #16407
• #16447 #16448 Fix an issue where create topics responses would show incorrect partition count and replication factor by @oleiman in #16449
• #16479 Fix timequery error that triggered full partition scan by @Lazin in #16520
• PR #16435 [v23.2.x] Fixed large allocation in kafka::wait_for_leaders by @mmaslankaprv

Improvements

• Improved handling of follower fetching offset validation when used with relaxed consistency by @mmaslankaprv in #16530
• Improves observability by allowing Redpanda to detect that some internal processes are stuck. by @Lazin in #16477
• #16423 Publish total reclaimable space to avoid stuck decommission scenario. by @dotnwat in #16442
• PR #16219 [v23.2.x] Introduced partition shutdown watchdog timer by @mmaslankaprv
• PR #16494 [v23.2.x] Wrapped logging with vlog macro in places that missed it by @mmaslankaprv

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.24...v23.2.25

Bug Fixes

• Fix the starter code for Rust projects in rpk transform init by @rockwotj in #16194
• Report runtime public metrics by task queue for all cores, not just core 0 by @rockwotj in #16203
• #16271 Fixes a bug that would previously cause read replicas to report the wrong value for the redpand_kafka_max_offset metric. by @andrwng in #16272

Improvements

• smaller memory footprint when using with large number of topics with small partition count by @mmaslankaprv in #16266
• PR #16182 [v23.3.x] Rename max_client_count to max_connection_count by @travisdowns
• PR #16195 [v23.3.x] Introduce wasm::logger by @oleiman
• PR #16216 [v23.3.x] Introduced partition shutdown watchdog timer by @mmaslankaprv
• PR #16236 [v23.3.x] c/topics_dispatcher: do not guesstimate leader ids by @mmaslankaprv
• PR #16255 [v23.3.x] admin api: skip partition info in /brokers end point by @bharathv

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.3.3...v23.3.4

Bug Fixes

• #16274 Fixes a bug that would previously cause read replicas to report the wrong value for the redpand_kafka_max_offset metric. by @andrwng in #16275
• PR #16221 [v23.2.x] Fixed skipping application of raft snapshot by @mmaslankaprv
• PR #16239 [v23.2.x] c/topics_dispatcher: do not guesstimate leader ids by @mmaslankaprv
• PR #16282 [v23.2.x] cloud_storage: hold gate in hydration by @andrwng

Improvements

• smaller memory footprint when using with large number of topics with small partition count by @mmaslankaprv in #16267
• PR #16254 [v23.2.x] admin api: skip partition info in /brokers end point by @bharathv

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.23...v23.2.24

Features

• spillover manifests are enabled by default for clusters that did not explicit set a value or null by @andijcr in #16174

Bug Fixes

• PR #16178 [v23.3.x] c/log_eviction_stm: do not request snapshot if already progressed by @mmaslankaprv
• Fix internal RPC client connection stall after more than 2^32 requests are sent. by @ztlpn in #16176
• Fix large allocation in partition manifest. by @dotnwat in #16188
• Fix tiered-storage housekeeping problem that may cause replaced segments to pile up if the spillover is enabled. by @Lazin in #16167
• Fix tiered-storage housekeeping problem that may cause replaced segments to pile up if the spillover is enabled. by @Lazin in #16170
• Protect against a very rare scenario where after node restart, some of the partition replicas hosted on that node could not take part in leader elections. by @ztlpn in #16080
• #15811 Several additional metrics will have their "partition" label aggregated away (i.e., into a single series per remaining label set with no partition label, whose value is the sum of all input series with the same label set and different partition labels). This is already the default behavior for most metrics, but this change extends it to almost all remaining metrics. by @travisdowns in #16094
• #16093 Several additional metrics will have their "partition" label aggregated away (i.e., into a single series per remaining label set with no partition label, whose value is the sum of all input series with the same label set and different partition labels). This is already the default behavior for most metrics, but this change extends it to almost all remaining metrics. by @travisdowns in #16100
• fixed incorrect fetch offset validation by @mmaslankaprv in #16169

Improvements

• Add a dedicated CPU scheduling policy for Data Transforms by @rockwotj in #16139
• Reduces the number of allocations performed by the auditing subsystem by @graphcareful in #16147
• rpk transform deploy --file now supports https:// URLs by @rockwotj in #16063
• PR #16111 [v23.3.x] archival: Start housekeeping jobs after STM sync by @Lazin
• PR #16103 [v23.3.x] Add at: in Top-N alloc site output by @travisdowns

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.3.2...v23.3.3

Bug Fixes

• Fix internal RPC client connection stall after more than 2^32 requests are sent. by @ztlpn in #16175
• Fix large allocation in partition manifest. by @dotnwat in #16191
• Fix tiered-storage housekeeping problem that may cause replaced segments to pile up if the spillover is enabled. by @Lazin in #16166
• Fix tiered-storage housekeeping problem that may cause replaced segments to pile up if the spillover is enabled. by @Lazin in #16171
• Have fetch handler ensure rack awareness is enabled before performing follower fetching by @michael-redpanda in #15914
• Protect against a very rare scenario where after node restart, some of the partition replicas hosted on that node could not take part in leader elections. by @ztlpn in #16082
• #15839 safer handle unknown properties in local state by @andijcr in #15874
• #15925 Prevent oversized allocs when group fetching from many partitions. by @rockwotj in #15926
• ext4 is no longer incorrectly detected as ext2 (all of ext2, 3 and 4 are assumed to be ext4). by @travisdowns in #15812
• fixed incorrect fetch offset validation by @mmaslankaprv in #16168
• PR #15816 [v23.2.x] c/archival_stm: do not reset _last_replicate on timeout by @nvartolomei
• PR #15989 [v23.2.x] r/offset_translator: remove unsafe bootstrap code by @ztlpn

Improvements

• #15829 Added new metric to provide Follower Fetching feature observability by @mmaslankaprv in #15831
• PR #16099 [v23.2.x] archival: Use explicit types to encode upload candidate creation result by @abhijat
• PR #16104 [v23.2.x] Add at: in Top-N alloc site output by @travisdowns

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.22...v23.2.23

Bug Fixes

• Fix an issue with Cargo.toml when initializing a Rust Data Transform project via rpk transform init by @rockwotj in #15947
• Fixes a crash if a WebAssembly function is deployed that immediately crashes. by @rockwotj in #15943
• Fixes an an improper initialization of metrics related to controller snapshot uploads. by @andrwng in #16074
• Have fetch handler ensure rack awareness is enabled before performing follower fetching by @michael-redpanda in #15915
• Prevent an assertion from being triggered when Wasm VMs fail immediately. by @rockwotj in #15941
• Redpanda used to accept an empty string in redpanda.rack in node config. This would cause issues in Kafka operations. Redpanda will now error on startup if redpanda.rack is set to an empty string. by @michael-redpanda in #15849
• Redpanda will now correctly handle an empty rack ID provided in a fetch request by @michael-redpanda in #15860
• #15928 Prevent oversized allocs when group fetching from many partitions. by @rockwotj in #15929
• ext4 is no longer incorrectly detected as ext2 (all of ext2, 3 and 4 are assumed to be ext4). by @travisdowns in #15855
• safer handle unknown properties in local state by @andijcr in #15838

Improvements

• Caches the connections local address preventing the need to make a system calls to grab this value when auditing events. by @graphcareful in #15958
• Data Transforms written in Golang now use a non-buffered write mechanism. Transforms that used to be written as by @rockwotj in #15936
• Support changing the timeout for WebAssembly functions by @rockwotj in #15984
• Support dynamically changing the limit for WebAssembly binary size by @rockwotj in #15984
• This PR partially reverts the change such that strict retention remains enabled after upgrade unless it had been explicitly disabled before the upgrade. by @dotnwat in #16084
• #15974 Internal kafka client now uses asynchronous compression (when possible) to reduce possibility of oversized allocations and reactor stalls by @michael-redpanda in #15976
• [rpk] more informative error message display on create topic failure by @michael-redpanda in #15847
• rpk transform deploy takes a --file flag to deploy a compiled WebAssembly binary. by @rockwotj in #15954
• PR #15843 [v23.3.x] Increase audit buffer sizes for audit scale test by @graphcareful
• PR #15866 [v23.3.x] rptest: log error on failure to delete bucket by @andrwng
• PR #15867 [v23.3.x] rptest: allow the new version of xfs/ext4 fs error msg by @nvartolomei
• PR #15871 [v23.3.x] c/s/leader_balancer: prevent oversized alloc by @rockwotj
• PR #15876 [v23.3.x] t/kgo: upgrade kgo to do a full run after /last_pass by @nvartolomei
• PR #15885 [v23.3.x] gh/workflow: add build message in promote trigger by @gousteris
• PR #15888 [v23.3.x] gha: s/git.ref_name/github.ref_name by @rockwotj
• PR #15930 [v23.3.x] Skip assertion in audit log tests if results beat the baseline by @graphcareful
• PR #15970 [v23.3.x] dt: Fixed flaky schemas test by @michael-redpanda
• PR #15980 [v23.3.x] tx_migration: avoid ping pong of requests between brokers by @bharathv
• PR #15988 [v23.3.x] r/offset_translator: remove unsafe bootstrap code by @ztlpn
• PR #15998 [v23.3.x] c/partition_balancer: use full partition move when disk is full by @mmaslankaprv
• PR #15999 [v23.3.x] securit/:OIDC: Enable licence check and telemetry by @BenPope
• PR #16065 [v23.3.x] cloud_storage: Improve scrubber by @Lazin
• PR #16073 [v23.3.x] archival: avoid division by 0 when computing slow down rate by @nvartolomei
• PR #16052 [v23.3.x] storage: enable space management by default by @dotnwat

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.3.1...v23.3.2

Bug Fixes

• Redpanda used to accept an empty string in redpanda.rack in node config. This would cause issues in Kafka operations. Redpanda will now error on startup if redpanda.rack is set to an empty string. by @michael-redpanda in #15848
• Redpanda will now correctly handle an empty rack ID provided in a fetch request by @michael-redpanda in #15861
• #15785 Fix a rare bug where http client connections would vanish from the connection pool leading to various operations hanging while waiting for an http client. by @nvartolomei in #15834
• PR #15873 [v23.2.x] c/s/leader_balancer: prevent oversized alloc by @rockwotj

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.21...v23.2.22

Features

• Add recovery mode - an option to start redpanda in "metadata-only" mode, skipping loading user partitions and allowing only metadata operations to enable recovery from fatal misconfiguration or resource exhaustion situations. Enabled by the recovery_mode_enabled node config property. by @ztlpn in #14236
• Add broker support for SASL reauthentication. To enable, config connections_max_reauth_ms > 0. by @oleiman in #13822
• Adds cloud storage scrubbing capabilities to Redpanda. In brief, the scrubber runs in the background and verifies the integrity of the cloud storage metadata and the existence of data referenced by it. When an issue is discovered, the redpanda_cloud_storage_anomalies will increment its counters based on the anomaly type. Per partition anomalies can be queried via the v1/cloud_storage/anomalies/ admin API endpoint. by @VladLazar in #13253
• Add per-shard download throughput limit to tiered-storage by @Lazin in #13552
• Added a Golang SDK for Data Transforms. by @rockwotj in #12322
• Added the ability do disable whole topics or specific topic partitions. Disabled partitions are ignored by Redpanda and producing/consuming to them is impossible. This is useful when only a handful of partitions prevent the cluster from becoming healthy (e.g. if the data in them is corrupted in a way that causes Redpanda to crash). by @ztlpn in #15141
• Adds an admin API to reset the crash loop prevention counter. Additionally the tracking metadata is reset every time the broker boots up in recovery mode. by @bharathv in #15064
• Adds node wise partition recovery functionality. Given an input set of node ids, force reconfigures all partitions that would lose majority if the specified set of nodes are dead. Intended to bulk recover partitions that are stuck when majority of brokers hosting their replicas are dead and irrecoverable. by @bharathv in #13943 and
  #15394
• Admin API: Support OpenID connect Authentication @BenPope in #14378
• Allow template prefix in advertised addresses by @RafalKorepta in #14177
• K8s: Allow to move pods into new nodes if needed, and allow down scaling Redpanda custom resource. by @alejandroEsc in #12847
• Force reconfiguration now supports recovering from an ‘all replicas lost’ scenario. The replicas are reset with empty logs (no data) on the (broker, shard) destinations passed in the reconfiguration command. by @bharathv in #13661
• HTTP Proxy: Support OpenID connect authentication by @BenPope in #14378
• Adds fast partition movement to speed up cluster scaling operations by @mmaslankaprv in #14305
• Implements audit logging capabilities for the admin server. To configure set audit_enabled to True and ensure one of the auditing enabled type in audit_enabled_event_types is set to management by @graphcareful in #14158
• Metadata backup: In addition to topic data, Redpanda will now upload cluster-related metadata, such as cluster configs, security settings, and consumer offsets into cloud storage. This metadata can be restored alongside topic data, allowing for the functional recovery of the entire cluster. by @andrwng in #15188
• Introduce Prometheus metrics for tracking TLS cert/CA expiration by @oleiman in #13477
• Kafka API: Adds SASL/OAUTHBEARER authentication to the Kafka API by @BenPope in #14378
• Redpanda is now compatible with Azure Storage Accounts that enable Hierarchical Namespaces. by @VladLazar in #12632
• K8s: Redpanda Node ID will be reported as label and annotation in Pod metadata by @RafalKorepta in #12524
• Schema Registry: Support OpenID connect by @BenPope in #14378
• You can decode now schema registry encoded messages (AVRO or Protobuf) using rpk topic consume --use-schema-registry by @r-vasquez in #14498
• #12912 Adds rpc_client_connections_per_shard cluster property that allows for the number of clients a broker opens to a given peer to be user configurable. by @ballard26 in #12906
• #12934 #13617 Time-based retention now uses broker-based timestamps for determining when to purge new data. This reduces the risk of retention not removing segments when a misbehaving client produces messages with incorrect timestamps (e.g. a timestamp in the future). by @andijcr in #12991
  • The configuration option storage_ignore_timestamps_in_future_secs is retained to deal with bad segments produced before v23.3
  • This changes the behavior for messages with a timestamp in the past. Before, retention would use this timestamp to delete data. now, the retention window starts when the message arrives in the broker.
• #12989 Schema Registry: Support GET /schemas/ids/{id}/subjects by @oleiman in #13020
• #13229 Schema Registry: Add DELETE /config/{subject} endpoint by @oleiman in #13557
• #14191 Audit Kafka API authentication and authorization events by @michael-redpanda in #14452
• #14394 rpk: adds rpk cluster partitions enable/disable: now users can disable or enable partitions of a topic using rpk. by @r-vasquez in #14909
• #14483 rpk debug bundle: now you can use the --partition flag to request additional debugging information from the admin API for the provided partitions. by @r-vasquez in #15136
• #9128 K8s: New Topic Custom Resource that can manage single topic by @RafalKorepta in #11208
• #9205 rpk: new 'rpk cluster partitions move' command to reassign replicas by @daisukebe in #13684
• #9205 rpk: new ‘rpk cluster partitions move-status’ to show ongoing partition movements by @daisukebe in #13258
• adds rpk redpanda mode recovery command to put redpanda into recovery mode. by @r-vasquez in #14431
• ability to change number of partitions in tx manager topic by @mmaslankaprv in #15121
• adds support to the Cluster API (deprecated) for a user-specified ServiceAccount by @joejulian in #12864
• adds support to the Console API (deprecated) for a user-specified ServiceAccount by @joejulian in #13120
• crash_loop_limit now defaults to 5. If a broker uncleanly shutdowns for 5 times back to back, it is considered to be in a crash loop mode and Redpanda refuses to start up and may need manual intervention. This enforcement is disabled in developer mode and rpk's dev-container mode. by @bharathv in #13431
• rpk introduces rpk cluster partitions list which lets users query cluster-level metadata of all partitions in the cluster. by @r-vasquez in #14862
• rpk now has a cluster txn command space by @twmb in #7557
• rpk now supports encoding messages with a given schema stored in the schema registry when producing via rpk topic produce. by @r-vasquez in #13543
• rpk: Introduce a new flag --label-selector to rpk debug bundle and have a way to filter the bundled resources by label. by @r-vasquez in #15185
• rpk: new rpk registry command to manage schema registry with rpk by @r-vasquez in #12669
• PR #14283 Compaction lossy hash by @dotnwat
• PR #14328 storage: implement sliding window compaction by @andrwng
• PR #14368 storage: utilities to implement sliding window compaction by @andrwng
• PR #14912 storage: use hash map in compaction by @andrwng
• PR #15269 storage: limit and account for memory used for compaction by @andrwng
• PR #15404 storage: further improvements to windowed compaction short circuit by @andrwng

Bug Fixes

• Fix a memory leak when transactions are used with many different producer IDs. by @rockwotj in #15796

• #15688 topic remote recovery via redpanda.remote.recovery handle correctly disabled retention by @andijcr in #15689

• #15786 Fix an issue where new configs would continually revert to legacy defaults after an upgrade. by @oleiman in #15787

• rpk: Redpanda log collection now is possible in pods where you have multiple containers, by default, rpk will gather logs from the 'redpanda' container. by @r-vasquez in #15684

• PR #15597 [v23.2.x] r/state_machine: ignore broken semaphore and condition var exceptions by @mmaslankaprv

• PR #15599 [v23.2.x] tests/e2e_shadow_indexing_test: wait until segments has one entry by @andijcr

• PR #15600 [v23.2.x] archival: Disable cross term merging of adjacent segments by @Lazin

• PR #15602 [v23.2.x] storage/disk_log_impl: defensive reverse iteration of _segs by @andijcr

• PR #15772 [v23.2.x] c/members_member: don't ignore update_broker_client future by @rockwotj

• PR #15780 [v23.2.x] Fix 128K iobuf zero-copy by @travisdowns

• PR #15800 [v23.2.x] c/topic_table: do not log duplicated lifecycle marker command by @mmaslankaprv

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.20...v23.2.21

Bug Fixes

• Fixed-limit truncation for request/response logging in the Kafka API by @oleiman in #15621
• Fixes a bug in compacted segment reuploads that could result in overlapping remote segments in the cloud manifest. by @andrwng in #15152
• PR #15682 [v23.2.x] r/persisted_stm: remove persistent state from previous incarnations by @ztlpn
• PR #15686 [v23.2.x] c/partition: fix acquiring _archiver_reset_mutex by @ztlpn
• PR #15699 [v23.2.x] codeowners: update consistency tests codeowner by @r-vasquez
• PR #15732 [v23.2.x] cloud_storage: Make wait for hydration abortable by @andrwng

Improvements

• Prevent metadata oversized allocations when requesting many topics. by @rockwotj in #15776

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.19...v23.2.20

Bug Fixes

• Fixes a bug in read replicas that were subject to unstable leadership that could create corrupted local segment files (note, segments in the cloud are safe). by @andrwng in #14634
• #14800 Fix a bug where producer would get INVALID_PRODUCER_ID_MAPPING if the leader of the transaction coordinator partition would change. by @nvartolomei in #14802
• #15048 Adds previously missing authorization checks to Transactions API by @oleiman in #15060
• #15239 Fix NotFound error handling when using Google Cloud Storage backend for Tiered Storage. by @nvartolomei in #15241
• #15280 Fixes an issue where lookup would fail for URL encoded username parameter (DELETE/PUT /v1/security/users/{user}) by @oleiman in #15283
• #15433 kafka/client: More robust error handling during initial connection by @BenPope in #15434
• prevents consumer from reading not majority acknowledged data by @mmaslankaprv in #15330

Improvements

• Avoid 100% reactor utilization in case of state machine errors. by @ztlpn in #15286
• Schema Registry: Improve compatibility of reading schemas that were created by another registry. by @BenPope in #14876
• #14833 Redpanda: Lower the log level if GNUTLS_E_DECRYPTION_FAILED is encountered. by @BenPope in #14869
• #15275 CreatePartitionsAPI responds with REASSIGNMENT_IN_PROGRESS when servicing a request for a topic that also has an active partition reassignment. by @NyaliaLui in #15276
• #15588 kafka/client: Configuration for consume min and max bytes by @BenPope in #15590
• redpanda: Make redpanda_cpu_busy_seconds_total metric a counter instead of a gauge. by @BenPope in #14928
• PR #14293 [v23.1.x] Fix for CI Failure (invalid timer interval) in UsageTest.test_usage_metrics_collection by @graphcareful
• PR #14497 [v23.1.x] Use trunk for promote job by @nk-87
• PR #14613 [v23.1.x] cmake: exclude operator tags when determining version by @rockwotj
• PR #14753 [v23.1.x] dt/memory_stress_test: Mark test_fetch_with_many_partitions @ok_to_fail by @michael-redpanda
• PR #14902 [v23.1.x] Remove k8s operator code from monorepo by @nk-87
• PR #14932 [v23.1.x] tests: disable minio console by @jcsp
• • PR #15157 Cleanup k8s operator code by @nk-87
• PR #15387 Revert "[v23.1.x] redpanda: Make redpanda_cpu_busy_seconds_total a counter" by @rockwotj

Full Changelog*: https://github.com/redpanda-data/redpanda/compare/v23.1.20...v23.1.21

Bug Fixes

• #15331 Speeds up convergence of leadership metadata in Redpanda by @mmaslankaprv in #15333
• #15435 kafka/client: More robust error handling during initial connection by @BenPope in #15436

Improvements

• Modifications to avoid stale responses returned from DescribeACLs requests by @graphcareful in #15437
• #15358 Improves space management by helping to reduce bias towards certain partitions when reclaiming space. by @dotnwat in #15359
• #15587 kafka/client: Configuration for consume min and max bytes by @BenPope in #15589
• PR #15246 [v23.2.x] remote_partition_fuzz_test: measure shutdown time directly by @andijcr
• PR #15380 [v23.2.x] rpk produce: Adjust timeout and maxBytes options by @r-vasquez
• PR #15389 [v23.2.x] rptest: Ignore transient log error message by @Lazin
• PR #15419 [v23.2.x] c/partition_balancer: handle members_table update during planning by @ztlpn
• PR #15421 [v23.2.x] tests: fix FollowerFetchingTest.test_with_leadership_transfers by @ztlpn

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.18...v23.2.19

Features

• #15074 rpk introduces rpk cluster partitions list which lets the user query the list of the partitions for a topic. by @r-vasquez in #15119
• #15235 rpk: add 'partitions move' to reassign replicas by @daisukebe in #15236

Bug Fixes

• Fixes an issue where adjacent segment compaction concatenates with an empty index resulting in incorrect concatenated segment. by @bharathv in #15372
• Prevent data balancing from scheduling partition moves back and forth when rack awareness and preventing full disk are in conflict (the latter gets a priority). by @ztlpn in #15292
• #15047 Adds previously missing authorization checks to Transactions API by @oleiman in #15061
• #15173 #15367 Fix a bug that resulted in offset_out_of_range errors for valid fetch offsets when fetching from followers that had recently lost leadership. by @ztlpn in #15369
• #15238 Fix NotFound error handling when using Google Cloud Storage backend for Tiered Storage. by @nvartolomei in #15240
• #15279 Fixes an issue where lookup would fail for URL encoded username parameter (DELETE/PUT /v1/security/users/{user}) by @oleiman in #15282

Improvements

• Avoid 100% reactor utilization in case of state machine errors. by @ztlpn in #15259
• #10963 Expose metric for total sent/received bytes from clients on public_metrics by @StephanDollberg in #15315
• #13992 #15080 #14829 #14820 easier to understand transaction related logs by @mmaslankaprv in #15139
• #14826 Adds a metric to track fetch plan and execute latency by @StephanDollberg in #15129
• #15173 #15367 more information when investigating offset out of range errors by @mmaslankaprv in #15369
• #15272 CreatePartitionsAPI responds with REASSIGNMENT_IN_PROGRESS when servicing a request for a topic that also has an active partition reassignment. by @NyaliaLui in #15274
• PR #14903 [v23.2.x] Remove k8s operator code from monorepo by @nk-87
• PR #15105 [v23.2.x] c/topic_table: fix force_update replicas_revisions update by @ztlpn
• PR #15125 raft/state_machine: log error for unknown exception by @andijcr
• PR #15126 [v23.2.x] tests/cluster_config_test: _check_value_everywhere retry and better error message by @andijcr
• PR #15139 [v23.2.x] Backport of #13964 #14739 #14803 #15049 #15032 by @mmaslankaprv
• PR #15148 [v23.2.x] bytes/io_iterator_consumer: add backtrace to std::out_of_range error by @andijcr
• PR #15156 Cleanup k8s operator code by @nk-87
• PR #15166 [v23.2.x] Feat/partition manifest log path by @andijcr
• PR #15175 Backport pr 14434 v23.2.x 808 by @graphcareful
• PR #15177 [v23.2.x] archival/test: Ensure archiver is stopped by @abhijat
• PR #15204 [v23.2.x] format exception fixes by @andijcr
• PR #15329 [v23.2.x] segment_meta_cstore: further restriction on hints when materializing the iterator by @andijcr
• PR #15345 [v23.2.x] Simplified mapping transaction id to coordinator partition by @mmaslankaprv
• PR #15364 [v23.2.x] rpk: manually update golangci-lint version by @r-vasquez
• PR #15384 Revert "[v23.2.x] redpanda: Make redpanda_cpu_busy_seconds_total a counter" by @rockwotj

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.17...v23.2.18

Features

• Adds rpc_client_connections_per_shard cluster property that allows for the number of clients a broker opens to a given peer to be user configurable. by @ballard26 in #14907

Bug Fixes

• Fixed transaction status being incorrectly reported when listing transactions via Kafka API by @mmaslankaprv in #14886
• Fixes a bug in segment reupload that could cause the operation to fail after a delete-records request. by @andrwng in #14839
• #15087 Fix a rare bug causing compaction to fail when lz4 compression is in use with lz4f_compressupdate error:ERROR_dstMaxSize_tooSmall. by @nvartolomei in #15088
• prevents consumer from reading not majority acknowledged data by @mmaslankaprv in #14918
• PR #14881 [v23.2.x] Add proper exception handler to around eviction_stm's call to replicate by @graphcareful
• PR #14981 [v23.2.x] Fixed handling Raft snapshot by @mmaslankaprv
• PR #14997 [v23.2.x] c/archival_stm: translate sync error to not_leader error code by @mmaslankaprv
• PR #15008 [v23.2.x] Better error message when topic creation fails by @travisdowns
• PR #15025 storage: fix variable name typo in gc logging statement by @dotnwat

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.16...v23.2.17

Bug Fixes

• #14799 Fix a bug where producer would get INVALID_PRODUCER_ID_MAPPING if the leader of the transaction coordinator partition would change. by @nvartolomei in #14801
• removed assumptions of no gaps and alignment from segment_meta_cstore by @andijcr in #14954
• PR #14851 [v23.2.x] cloud_storage_clients: Fix xml parsing when searching for error code in response by @abhijat
• PR #14934 [v23.2.x] tests: retry on timeout in partition balancer test by @mmaslankaprv
• PR #14944 [v23.2.x] rpk: add no-browser to cloud login by @r-vasquez
• PR #14952 [v23.2.x] cloud_storage: Add client address to fetch and log reader config by @abhijat
• PR #14958 [v23.2.x] cloud_storage: Log client address in batch parser by @abhijat

Improvements

• Schema Registry: Improve compatibility of reading schemas that were created by another registry. by @BenPope in #14875
• #14832 Redpanda: Lower the log level if GNUTLS_E_DECRYPTION_FAILED is encountered. by @BenPope in #14868
• redpanda: Make redpanda_cpu_busy_seconds_total metric a counter instead of a gauge. by @BenPope in #14926
  Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.15...v23.2.16

Bug Fixes

• Fixed a bug in which partition movement wouldn't take into account reclaimable space, preventing data movement by @mmaslankaprv in #14304
• #13753 Fix a bug where archival would not sync it's state correctly before proceeding with new segment uploads.
• #14210 fixes incorrect reporting of maintenance statues if a node is stopped by @mmaslankaprv in #14211

Improvements

• Add a new admin API endpoint which can reset the in-memory cloud storage partition metadata the latest uploaded one in cloud storage: /v1/cloud_storage/unsafe_reset_metadata_from_cloud/... by @VladLazar in #14598
• #14504 Improved Tiered-Storage metadata consistency by @Lazin in #14580
• PR #14408 [v23.2.x] Use the official buildkite plugin to trigger the promotion by @nk-87
• PR #14476 [v23.2.x] tests: do not reserve disk space in full_node_test by @mmaslankaprv
• PR #14496 [v23.2.x] Use trunk for promote job by @nk-87
• PR #14614 [v23.2.x] cmake: exclude operator tags when determining version by @rockwotj
• PR #14617 [v23.2.x] Fix for CI failure (failure to assert_usage_consistency) in UsageTest.test_usage_collection_restart by @graphcareful
• PR #14639 [v23.2.x] tests/htt: import missing regex package that breaks tests by @gousteris
• PR #14650 [v23.2.x] tests/topic_recovery_tests: archiver_term added to replaced segment key by @andijcr
• PR #14738 [v23.2.x] ducktape/cloud_storage: Ignore accessdenied errors by @abhijat
• PR #14755 [v23.2.x] dt/memory_stress_test: Mark test_fetch_with_many_partitions @ok_to_fail by @michael-redpanda
• PR #14771 [v23.2.x] cloud_storage: Use abort source when waiting for semaphore units by @Lazin
• PR #14790 [v23.2.x] ducktape: Update librdkafka and confluent-kafka to v2.2.0 by @andrewhsu

Full Changelog: https://github.com/redpanda-data/redpanda/compare/v23.2.14...v23.2.15