v2.4.3.0 - May 12, 2021

Downloads

Docker

docker pull yugabytedb/yugabyte:2.4.3.0-b6

Improvements

Core database

• [8052] Add ability to configure cipher list

Yugabyte Platform

• [1342] Fixing the error message when the get host info call to gcp fails

• [8165]Yugaware creates zombie ssh process

Bug fixes

• [8176] GCP universes don't have correct nproc system settings.

Core database

• [8150] [8196] Fix preceding op id in case of empty ops sent to the follower

• [8348] Correctly handling a failure to create a priority thread pool worker thread

Yugabyte Platform

• [6758] Unable to reuse On-premise instance incase previouse universe failed / deleted (#8263)

v2.7.0 - May 5, 2021

Yugabyte release 2.7.0 builds on our work in the 2.5 series, which fed into the 2.4 stable release. With release 2.7.0, we're planning development on a number of new features, as well as refinements to existing functionality.

Build: 2.7.0.0-b17

Downloads

Docker

docker pull yugabytedb/yugabyte:2.7.0.0-b17

New Features

Yugabyte Platform

• Yugabyte Platform High Availability is now available in BETA. With this feature, you can deploy multiple platforms in an active-standby cluster, ensuring you can always monitor and manage your platform-managed universes regardless of outages to any particular platform in the HA cluster.
• [7225] [7228] [UI] Enable the OpenShift tab, marked as beta

Core Database

• [2717] YSQL: Support ALTER FUNCTION command
• [6985] YSQL: Add simple UNIQUE column
• Point in time restore:
  • [7121] Extend yb-admin restore_snapshot to use a custom time
  • [7126] PITR: Introduce snapshot schedule

Improvements

Yugabyte Platform

• [5535] [Platform] Remove stale instance configs after cloud provider is deleted (#6975)
• [5624] Show pricing info for Azure
• [5626] Enable Hosted Zone for Azure
• [5628] [Azure] Support regions with no availability zones
• [5807] Add toggle to show deleted backups
• [5838] [Platform] Add node link to corresponding Azure portal URL
• [5840] Add user tags for Azure universes
• [5841] Show only Azure VMs that meet minimum requirements
• [6321] [Platform] Support using Shared Gallery Images when creating Azure universes
• [6681] [Platform] Show kubectl command for Kubernetes pods in Connect modal (#7506)
• [6712] Fix issue with JDK incompatibility in Java RPC client
• [6756] [Platform] Created date showing up as "Invalid Date". (#7158)
• [7024] [Platform] Unable to edit number of nodes in AZ section (#7350)
• [7054] [YW] Add conditional checks for hiding specific platform elements in non-platform mode.
• [7372] [Platform] Skip running periodic schedules when in follower mode
• [7433] [Platform] Standby instances backup time not consistent after restoring active instance
• [7443] [Platform] Fixed live query details side panel doesn’t go away after unchecked
• [7445] [Platform] Add pagination for slow queries
• [7472] [Platform] Hide the Upgrade button from info card on Pause universe. (#7504)
• [7475] Use more recent CentOS-7 base image for GCP universe VMs
• [7493] [Platform] Menu shows 0 appended to "Upgrade Software0" - Removed 0
• [7548] [Platform] Set versions for google modules in requirements.txt
• [7549] [Platform] Platform uses public IP instead of private IP to connect to Azure universes
• [7576] Ensure rsync is available on latest GCP image
• [Platform] Use more recent CentOS-7 base image for GCP universe VMs #7475
• [Platform] parsing of df output is fragile and may fail in case of "safe" error in df #7402

Core Database

• [4944] YSQL performance improvements in TTL-related computations
• [5922] YCQL: Improve audit logging
• [7359] YSQL: Support adding primary key to a table with tablespace
• [7366] YSQL: Allow getting current SQL query in pggate for debug logging
• [7404] YSQL: Extend CHECKPOINT to have beta noop functionality
• [7418] [7463] YSQL: Import the 'force' option for the Drop Database command
• [7532] Master performance improvements: reduce the scope of catalog manager state lock in ScopedLeaderSharedLock constructor

Bug Fixes

Core Database

• [5383] YSQL bug fixes for Jepsen

Known Issues

Yugabyte Platform

N/A

Core Database

N/A

Notes

{{< note title="New release versioning" >}}

Starting with v2.2, Yugabyte release versions follow a new release versioning convention. The latest release series, denoted by MAJOR.ODD, incrementally introduces new features and changes and is intended for development and testing only. Revision releases, denoted by MAJOR.ODD.REVISION versioning, can include new features and changes that might break backwards compatibility. For more information, see Supported and planned releases.

{{< /note >}}

v2.4.2.0 - May 03, 2021

Downloads

Docker

docker pull yugabytedb/yugabyte:2.4.2.0-b45

Improvements

Core database

• [5647] docdb: LB should not move tablets right after splitting
• [6241] YSQL: Make row key share lock less restrictive
• [6305] Adaptive Heartbeat Reporting
• [6445] docdb: Master should rebuild YCQL system.partitions on a background thread
• [7079] [7153] YSQL: Improve configuration and defaults for client to server encryption
• [7355] ysql: check backfill bad connection status
• [7455] [YCQL] Update index from transaction with cross-key statements.
• [7641] YCQL: Fix checks in index update path that determine full row removal.
• [7705] ysql: prioritize internal HBA config
• [7813] [YSQL] YSQL dump should always include HASH/ASC/DESC modifier for indexes/pkey.

Yugabyte Platform

• [6668] Support platform backup/restore on Kubernetes

Bug fixes

Core database

• [5383] Fix conflict resolution while adding read intents
• [6747] [6902] [7109] [7284] YCQL system.partition bug fixes
• [6960] docdb - Register ScopedRWOperation when accessing doc_db data in StillHasParentDataAfterSplit
• [7105] Explicitly clear the global PgMemctx map in YBCDestroyPgGate
• [7145] Fix wrong propagated safe time
• [7397] Tablet splitting: Crash during generating TSHeartbeat report after split
• [7835] Don't crash when trying to append ValueType::kTombstone to a key

Yugabyte Platform

• [1342] [Platform] Fixing the error message when the get host info call to cloud providers fails
• [6698] [7262] Platform: Cascade delete alert and alert_definition
• [7054] [Platform] Add conditional checks for hiding specific platform elements in non-platform mode.
• [7268] [Platform] Use the correct SSH user for on-prem node preflight checks
• [7385] [Platform] Missing package for s3cmd dependency for airgapped GCP VMs
• [7390] [Platform] Preflight checks should handle sudo passwords when given
• [7402] [Platform] parsing of df output is fragile and may fail in case of "safe" error in df
• [7408] [Platform] Insufficient user feedback in platform for Task Retry
• [7416] [Platform] Changed default port of On-Prem provider to 22 (#7599)
• [7475] [Platform] Use more recent CentOS-7 base image for GCP universe VMs
• [7597] [Platform] Add python modules to PYTHONPATH in non-replicated installations
• [7656] [Platform] Use the correct SSH user + port for onprem workflow.
• [7698] [Platform] Custom SMTP Configuration API returns unmasked SMTP password
• [7703] [Platform] Can't send email for custom SMTP settings without authentication (empty username)
• [7704] [Platform] Backup to S3 fails using Platform instance's IAM role
• [7769] Prevent adding on-prem node instance with duplicate IP
• [7771] [Platform] A lot of exceptions in logs "Error querying for alerts..."
• [7780] [Platform] Make backup deletion independent of schedule
• [7810] [Platform] Update SMTP configuration for health check emails
• [7909] [Platform] Fix issue with signature could not be verified for google-cloud-sdk for GCP VMs
• [7918] [Platform] Add US-West2 GCP metadata to Platform
• [7988] [Platform] Backup deletion script shouldn't cause retries

v2.5.3.1 - Mar 16, 2021

Build: 2.5.3.1-b10

Downloads

Docker

docker pull yugabytedb/yugabyte:2.5.3.1-b10

New Features

Yugabyte Platform

• Yugabyte Platform High Availability is now available in BETA. With this feature, you can deploy multiple platforms in an active-standby cluster, ensuring you can always monitor and manage your platform-managed universes regardless of outages to any particular platform in the HA cluster.
• [7225] [7228] [UI] Enable the OpenShift tab, marked as beta

Core Database

• [2717] YSQL: Support ALTER FUNCTION command
• [6985] YSQL: Add simple UNIQUE column

Improvements

Yugabyte Platform

• [5628] [Azure] Support regions with no availability zones
• [5807] Add toggle to show deleted backups
• [6373] Pause/Resume universe
• [6962] Expose endpoint for downloading log files
• [7475] Use more recent centos-7 base image for GCP universe VMs

Core Database

• [7121] Extend yb-admin restore_snapshot to use a custom time
• [7126] PITR: Introduce snapshot schedule
• [7366] YSQL: Allow getting current SQL query in pggate for debug logging
• [7418] [7463] YSQL: Import the 'force' option for the Drop Database command

Bug Fixes

Yugabyte Platform

• [4742] Plotly tooltip is overlaping with no data on zero pozitions (on x or y)
• [6213] Fixed poor visual feedback for the node staus change
• [6530] Enforced better password strength for registration
• [6611] [Kubernetes] Consider AZ level namespace for the metrics
• [6718] Health check emails should include Yugaware Hostname
• [7024] Unable to edit number of nodes in AZ section
• [7149] Trigger trying to take a platform backup when a task has completed
• [7161] Fix loading instance types on AZ change and inability to edit volume size
• [7234] UI fixes for platform
• [7363] [UI] Certificate list improvements
• [7392] Fix the cqlsh healthcheck failures happening because of warnings in the output
• [7468] Fixed issues of deleting a backup goes to "unknown" state for that backup
• [7490] Ensure prom config is reverted to non-federated when deleting a follower HA config
• [7496] Alerts should not be triggered for a paused universe when running health check
• User menu not clickable in Platform UI

Core Database

• [2865] YSQL: Speed up DROP DATABASE by batching tablet deletion into one raft request
• [5543] YSQL: unable to alter column with correct options after a previous attempt with incorrect options fails
• [6183] YSQL: Import Fix memory leak in plpgsql's CALL processing
• [6241] YSQL: Make row key share lock less restrictive
• [6253] YCQL: Create Index does not have to wait for the backfill to complete
• [7065] YCQL: Fixed incorrect CQL response in case of IN() with no argument
• [7380] YSQL: Import Fix another ancient bug in parsing of BRE-mode regular expressions
• [7386] YSQL: Import Fix ancient bug in parsing of BRE-mode regular expressions
• [7397] Tablet splitting: Crash during generating TSHeartbeat report after split
• [7400] YSQL: Prevent drop of database when there is active connection to the DB
• [7413] DocDB: Fixed MetaCache::ProcessTabletLocations to reuse partitions list for co-located tables
• [7480] Remove reference to user from YSQLsh health check output

Known Issues

Yugabyte Platform

7493 Menu shows 0 appended to "Upgrade Software0" when there is nothing to upgrade. When the universe is up-to-date (in other words, there are no upgrades available), the Action menu shows "Upgrade Software0". This is a cosmetic bug, and will be addressed in a future release.

New Features

Yugabyte Platform

• [7126] PITR: Introduce snapshot schedule
• [7225] [7228] [Platform] [UI] Enable the OpenShift tab, marked as beta

Core Database

• [2717] YSQL: Support ALTER FUNCTION command
• [6985] YSQL: Add simple UNIQUE column

Improvements

Yugabyte Platform

• [5628] [yugaware] [azure] Support regions with no availability zones
• [5807] [Platform] Added toggle to show deleted backups
• [6373] Platform: Pause/Resume universe. (#6846)
• [6962] Platform: Exposes endpoint for downloading log files.
• [7121] Extend yb-admin restore_snapshot to use a custom time

Core Database

• [7366] YSQL: Allow getting current SQL query in pggate for debug logging
• [7418] [7463] YSQL: Import Introduce the 'force' option for the Drop Database command.

Bug Fixes

Yugabyte Platform

• [4742] [Platform] Plotly tooltip is overlaping with no data on zero pozitions (on x or y)
• [5383] Fix conflict resolution while adding read intents
• [6213] [Platform] Fixed poor visual feedback for the node staus change
• [6530] [Platform] Enforced better password strength for registration
• [6611] [Platform] [Kubernetes] Consider AZ level namespace for the metrics (#7479)
• [6718] [Platform]: Health check emails should include Yugaware Hostname (#7337)
• [7149] Platform: Trigger trying to take a platform backup when a task has completed
• [7161] Platform UI: fix loading instance types on AZ change and inability to edit volume size
• [7234] [Platform] UI fixes for platform
• [7342] [Platform] [UI] add Beta tag to HA tab + address feedback
• [7363] [Platform] [UI] Certificate list improvements
• [7392] [Platform] Fix the cqlsh healthcheck failures happening because of warnings in the output (#7364)
• [7423] Platform: Fix AvailabilityZone::getByCode
• [7468] [Platform] Fixed issues of deleting a backup goes to "unknown" state for that backup
• [7488] Platform: Fix a sync backup bug due to File::renameTo behavior
• [7490] Platform: Ensure prom config is reverted to non-federated when deleting a follower HA config
• [7494] [YW] Hide Pause Universe in universe actions dropdown
• [7496] Platform: Alerts should not be triggered for a paused universe when running health check

Core Database

• [2865] YSQL: Speed up DROP DATABASE by batching tablet deletion into one raft request
• [5543] YSQL: unable to alter column with correct options after a previous attempt with incorrect options fails
• [6183] YSQL: Import Fix memory leak in plpgsql's CALL processing
• [6241] YSQL: Make row key share lock less restrictive
• [6253] YCQL: Create Index does not have to wait for the backfill to complete
• [7065] [YCQL] Fixed incorrect CQL response in case of IN() with no argument.
• [7380] YSQL: Import Fix another ancient bug in parsing of BRE-mode regular expressions.
• [7386] YSQL: Import Fix ancient bug in parsing of BRE-mode regular expressions.
• [7397] Tablet splitting: Crash during generating TSHeartbeat report after split
• [7400] YSQL: Prevent drop of database when there is active connection to the DB
• [7413] docdb: Fixed MetaCache::ProcessTabletLocations to reuse partitions list for co-located tables
• [7480] Remove reference to user from YSQLsh health check output

v2.5.2 - Feb 23, 2021

Build: 2.5.2.0-b104

Downloads

Docker

docker pull yugabytedb/yugabyte:2.5.2.0-b104

Improvements

Yugabyte Platform

• [5376] Added EULA in user registration flow during Platform installation
• [5732] Disable "Delete provider" for on-premise provider with active universes
• [5836] Package managed python packages in a portable manner
• [5919] Platform: change default storage option for AWS provider to 1x250GB
• [6017] Platform: Need ability for platform to check for actual clock synchronization
• [6035] Platform UI: show region level config paths for k8s providers
• [6084] Platform UI: disable STOP and REMOVE actions for kubernetes pods
• [6165] Platform: Renamed "Universe backups" label for keyspace -or- namespace level backups.
• [6230] Platform: Add tooltips to KMS Configuration UI
• [6346] UI visual feedback for KMS related tasks
• [6421] Edit Universe fails silently in case when it has decommissioned node(s)
• [6477] UI validation for duplicate region codes
• [6540] Add ability to remove certificates from UI
• [6655] Platform: Enabled AWS timeSync as a default on AWS universe
• [6680] [6882] Platform: Disabled deleting storage configuration when it is in use
• [6708] For AWS universes fetch instance types with respect to selected availability zones
• [6740] Update clients to support mTLS in YB clusters.
• [6807] Platform: Kubernetes: Allow specifying namespace in the AZ config
• [6838] [6839] [YW] Remove YSQL transactions metrics from Metrics tab
• [6858] Platform: Preflight checks for NFS backup storage configuration
• [6893] Platform: Allow platform to specify min TLS version for its own frontend

Core Database

• [1104] YSQL: ALTER TABLE ADD PRIMARY KEY
• [3565] tools: Allow developers to run yugabyted from the repo
• [4230] tools: Multi-node cluster restart support for yugabyted
• [5686] YSQL: apply empty deletes sparingly
• [5716] Enable Master Register From Raft by Default
• [6128] 2dc: Enable Replicate Intents By Default
• [6215] YSQL: Support index backfill on colocated tables
• [6511] YSQL: the WITH clause is now supported
• [6571] Handle read replicas when checking counts in GetTabletLocations
• [6632] ybase: Add new flag for how many servers to wait for before the transactions table is
• [6636] docdb: Cache table->tablespace->placement information in YB-Master
• [6637] YSQL: Support CREATE TABLE/INDEX <table/index> SET TABLESPACE
• [6638] YSQL: Support DROP TABLESPACE API
• [6720] YSQL: pg_hint_plan is now available. This allows you to adjust the query execution plan with hinting phrases in specially formatted comments.
• [6749] YSQL: Add support for FETCH NEXT, FORWARD, and positive row-count options
• [6771] YSQL: Add functionality to validate keys in a JSON object
• [6772] Display tserver clock information in yb-master UI
• [6784] YSQL: use all rather than 0.0.0.0/0, ::0/0 in hba conf (#7046)
• [6825] Add tablet server readiness to yb-ts-cli
• [6833] TLS Improvements: Add flag to skip client endpoint verification
• [6841] Display table/cluster replication info in the table UI
• [6900] YSQL: Imported from PostgreSQL: Avoid some table rewrites for ALTER TABLE .. SET DATA TYPE timestamp
• [6928] [7105] Add tracing support for yb-client/transactions
• [6966] YCQL: Java driver changes to allow primary key column with DECIMAL type.
• [6996] ybase: List the host for pending remote bootstrap
• [7015] Allow providing restore time for snapshot
• [7016] docdb: Enabled sanity check that tablet lookup result matches partition key
• [7038] ybase: Make callhome_enabled a runtime flag

Bug Fixes

Yugabyte Platform

• [5172] Health check time is always in the Pacific timezone
• [5489] Fix slow query panel returning TypeError from API request.
• [5489] [YW] Add tab panels to Queries tab to separate live from slow queries.
• [5571] Use local YSQL socket for backups if --ysql_enable_auth is true
• [5594] Platform: Handle invalid certs/keys correctly
• [5694] Platform: Fixed Metrics custom time window issue
• [6164] Manual backup of ysql task is not showing up in the universe tasks page
• [6320] Platform: Create Universe does not validate encryption-at-rest
• [6583] Platform: Fixed the UI alignment for on-prem universes
• [6758] For on-prem universes, unable to reuse certain instances that are disassociated from a universe
• [6759] [6760] [6814] Platform: Fixed availability zone error scenarios
• [6817] [6904] Restore of large number of keyspaces/tables fails with '413 Request Entity Too Large
• [6901] Platform: YW Fails to send alert email
• [6911] Platform: Alert for removed universe fires forever
• [6919] Fix crashing on-prem provider page due to missing instance type details
• [6965] YSQL backups with node-to-node TLS encryption enabled hang forever
• [6983] Increase ssh timeout for checking instance availability
• [7082] Platform: Health-check script generates an alert for 2.5.x when using Python 3
• [7107] Platform: Changing node filter for universe metrics does not update the graphs
• [7114] Platform: Fixes a bug while backing up multiple ysql namespaces for encrypted at rest universe
• [7159] Allow only one cluster per K8s provider with namespace
• [7196] Health checks should default to TLSv1.2
• [7233] Fix data format for slow query top sql statement data cells
• [7268] Use the correct SSH user for on-prem node preflight checks

Core Database

• [4599] YSQL: Fix calculation for transaction counts in YSQL metrics.
• [4813] YSQL: Imported from PostgreSQL: Don't log incomplete startup packet if it's empty
• [5007] YSQL: Properly set retain_delete_markers for colocated tables
• [6397] ybase: blacklisted TS' initial load not replicated during master failover
• [6406] docdb: Fixed SPLIT_OP replay during bootstrap of the tablet that was crashed before SPLIT_OP apply
• [6436] YSQL: Avoid cache invalidation initiated by other process
• [6537] YSQL: Backup: Fixed CREATE INDEX statement generated by ysql_dump to use SPLIT INTO syntax.
• [6593] Additional default firewall ports
• [6628] docdb: Read Table Partitions from Snapshot
• [6735] YCQL: Treat overwritten docdb collection entries as expired
• [6811] YSQL: apply empty deletes for index backfill
• [6829] [6879] YCQL: Fixed various issues for literals of collection datatypes
• [6876] 2dc: Fix data race between TwoDCOutputClient::WriteCDCRecordDone and TwoDCOutputClient destructor
• [6881] YSQL: Imported from PostgreSQL: Fix calculation of how much shared memory is required to store a TOC
• [6886] YSQL: Backup: Fixed backup-restore failure due to unexpected NULLABLE column attribute value.
• [6890] docdb: Fixed tablet splitting partitions version refresh
• [6902] Remove recently added LongOperationTracker from ThreadPool
• [6903] YSQL: Imported from PostgreSQL: Fix pg_dump for GRANT OPTION among initial privileges.
• [6943] YSQL: Imported from PostgreSQL: Fix ALTER DEFAULT PRIVILEGES with duplicated objects
• [6953] docdb: Intent metadata_write_time is incorrectly set
• [6960] docdb - Register ScopedRWOperation when accessing doc_db data in StillHasParentDataAfterSplit
• [7008] YQL: PermissionsManager::AlterRole() should always rebuild permissions
• [7040] 2DC: Fix Race Condition On Consumer With Smaller Batch Sizes
• [7070] YSQL: Imported from PostgreSQL: 'Fix ancient memory leak in contrib/auto_explain.'
• [7071] Update to latest version of cqlsh
• [7079] [7153] YSQL: Improve configuration and defaults for client to server encryption
• [7094] YSQL: Imported from PostgreSQL: 'Avoid crash when rolling back within a prepared statement.'
• [7097] YSQL: Allow table size hints to be used by query planner
• [7105] Explicitly clear the global PgMemctx map in YBCDestroyPgGate
• [7109] Fix a race in accessing the YQLPartitionsVTable::cache_ object
• [7151] YSQL: disallow tampering with postgres role
• [7192] YSQL: Imported from PostgreSQL: 'Fix usage of whole-row variables in WCO and RLS policy expressions.'
• [7237] YSQL: Imported from PostgreSQL: 'pg_attribute_no_sanitize_alignment() macro'
• [7241] YSQL: Imported from PostgreSQL: 'Avoid divide-by-zero in regex_selectivity() with long fixed prefix.'

Release notes

v2.2.7 - Feb 19, 2021

Downloads

Docker

docker pull yugabytedb/yugabyte:2.2.7.0-b5

Bug Fixes

Yugabyte Platform

• Backup-related tasks (schedules, restores, deletes) failed when the storage configuration was deleted 6680, 6882
• Fixed incorrect selection of the preferred leader zones 6743
• Removed the "Transactions" plot line from YSQL Ops and Latency graphs 6839
• Added ability to specify allowed (and disallowed) SSL protocols 6865

Core Database

• Fixed build issues on 2.2 due to Clang version 5458, 5812
• Avoid running out of threads due to FailureDetector 5752
• Fix backup-restore issues when source table was altered before 5958
• ybase: a blacklisted TS' initial load was not replicated during master failover 6397
• YCQL: Fixed range deletes to start and end at specified primary key bounds 6649
• YCQL: Fixed a bug with an index update on a list after an overwrite 6735
• YCQL: Fixed JSONB operator execution when applying to NULL objects 6766
• YCQL: Fixed parser typo when scanning binary values 6827
• YCQL: Fix case-insensitive pattern matching for audit logging 6836

v2.4.1 - Feb 17, 2021

Downloads

Docker

docker pull yugabytedb/yugabyte:2.4.1.0-b25

Improvements

Yugabyte Platform

• When creating/restoring backups, use the local UNIX socket to connect to YSQL instead of the hostname if authentication is enabled 5571
• Optimizations to theYSQL layer to apply empty deletes only when required 5686
• On universe creation, platform checks that clocks are synchronized 6017
• Health Check generates alerts if it isn't running 6581
• Manual provisioning now runs the same preflight checks as regular provisioning 6819
• Retrying a failed task now provides visual feedback 6820
• Removed the "Transactions" plot line from YSQL Ops and Latency graphs 6839
• You can now specify a minimum TLS version for Platform 6893, 7140
• Health checks now default to TLS 1.2 7196

Core Database

• The YB-TServer metrics output (for example, on localhost) now shows the total affected rows for each operation. 4600
• YSQL: Backup for colocated databases 4874
  • Backup: Fix restore of colocated table with table_oid already set 6678
• Metrics thread now start after the first replication stream is created 5251
• The YB-TServer metrics output (for example, on localhost) now shows transaction BEGIN, COMMIT, and ROLLBACK statements. 6486
• Restore now preserves the exact partitioning of the source tablets 6628
• Enabled a sanity check to ensure that the tablet lookup result matches the partition key 7016

Bug Fixes

Yugabyte Platform

• Release instance was not an option for a node if an install failed because of SSH access 5942
• Backup-related tasks (schedules, restores, deletes) failed when the storage configuration was deleted 6680
• VPC cross-linking failed during creation of an AWS provider 6748
• Fixes to YSQL backups with node-to-node TLS encryption enabled 6965
  • Add certs flags to ysql_dump when backing up a node-to-node TLS-enabled universe
• Corrected an error when backing up multiple YSQL namespaces in a universe that is encrypted at rest 7114
• Fixed a syntax error in replicated.yml 7180
• Fixed an issue preventing health checks from using an appropriate TLS version 7196

Core Database

• ybase: a blacklisted TS' initial load was not replicated during master failover 6397
• The CREATE INDEX statement generated by ysql_dump now uses SPLIT INTO syntax 6537
• YCQL: Fixed a bug with an index update on a list after an overwrite 6735
• Fixed a bug in YSQL for online index backfill capabilities, to ensure deletes to an index are persisted during backfill 6811
• YCQL: Fixed various issues for literals of collection datatypes 6829, 6879
• The permissions map is now rebuilt after all ALTER ROLE operations 7008
• Fixes to Async Replication 2DC capabilities, including a bug fix for a race condition on consumer with smaller batch sizes 7040

Release Notes

v2.4.0 - Jan 21, 2021

New Features

Yugabyte Platform

• Data encryption in-transit enhancements:

  • Support for bringing your own CA certificates for an on-premise cloud provider (#5729)
  • Support for rotating custom CA certificates (#5727)
  • Displaying the certificate name in the YB Platform Universe editor
  • Moving of the Certificates page to Cloud Config > Security > Encryption in Transit

• Moving of TLS In-Transit Certificates to Cloud > Security Config

• Support for Rolling restart of a universe (#6323)

• Support for VMware Tanzu as a new cloud provider (#6633)

• Alerts for backup tasks (#5556)

Core Database

• General availability of Automatic Tablet splitting, which enables resharding of data in a cluster automatically while online, and transparently to users, when a specified size threshold has been reached (#1004)
• Introducing support for audit logging in YCQL and YSQL API (#1331, #5887, #6199)
• Ability to log slow running queries in YSQL (#4817)
• Introducing support for LDAP integration in YSQL API (#6088)

Improvements

Yugabyte Platform

• Support for Transactional tables and Cross Cluster Async Replication topology (#5779)

• Support for very large transactions and stability improvements (#1923)

• Displaying an entire query in the detailed view of the live queries tab (#6412)

• Not returning Hashes and Tokens in API responses (#6388)

• Authentication of proxy requests against a valid platform session (#6544)

• Disabling of the unused API endpoint run_query (#6383)

• Improved error handling for user-created concurrent backup tasks (#5888)

• Performance improvements for live queries (#6289)

• Pre-flight checks for Create Universe, Edit Universe, and Add Node operations for an on-premise provider (#6016)

• Disabling of the unused API endpoint run_in_shell (#6384)

• Disabling of the unused API endpoint create_db_credentials (#6385)

• Input validation for Kubernetes configuration path (#6389)

• Input validation for the access_keys API endpoint (#6386)

• Input path validation for backup target paths (#6382)

• Timeout support for busybox (#6652)

• Enabling of CORS policy by default (#6390)

• Deleting backups for TLS-enabled universes (#5980)

Core Database

• DDL consistency (#4710, #3979, #4360)

• DDL performance improvements (#5177, #3503)

• ALTER versions for ORM support (#4424)

• Online index backfill GA (#2301, #4899, #5031)

• Table partitioning (#1126, #5387)

• SQL support improvements:

  • Support for the FROM clause in UPDATE (#738)
  • Support for the USING clause in DELETE (#738)
  • SQL/JSON path language [PG12] (#5408)

• Performance improvements:

  • Sequence cache min-value flag (#6041, #5869)
  • Foreign-key batching (#2951)

• YSQL usability improvements:

  • COPY FROM rows-per-txn option (#2855, #6380)
  • COPY / ysql_dump OOM (#5205, #5453, #5603)
  • Large DML OOM (#5374)
  • Updating pkey values (#659)
  • Restarting write-txns on conflict (#4291)

• YSQL statement execution statistics (#5478)

• Improved raft leader stepdown operations, by waiting for the target follower to catch up. This reduces the unavailability window during cluster operations. (#5570)

• Improved performance during cluster overload:

  • Improved throttling for user-level YCQL RPC (#4973)
  • Lower YCQL live RPC memory usage (#5057)
  • Improved throttling for internal master level RPCs (#5434)

• Improved performance for YCQL and many connections:

  • Cache for generic YCQL system queries (#5043)
  • Cache for auth information for YCQL (#6010)
  • Speedup system.partitions queries (#6394)

• Improved DNS handling:

  • Process-wide cache for DNS queries (#5201)
  • Avoid duplicate DNS requests during YCQL system.partitions queries (#5225)

• Improved master-level load balancer (LB) operations:

  • Throttle tablet moves per table (#4053)
  • New global optimization for tablet balancing -- this also allows the LB to work with colocation (#3079)
  • New global optimization for leader balancing (#5021)
  • Delay LB on master leader failover (#5221)
  • Register temporary TS on master failover to prevent invalid LB operations (#4691)
  • Configurable option to disallow TLS v1.0 and v1.1 protocols (#6865)

• Skip loading deleted table metadata into master memory (#5122)

Bug Fixes

Yugabyte Platform

• Edit Universe did not display the TLS certificate used
• Multi-zone K8s universe creation failed (#5882)
• Tasks page reported an incorrect status of a failed backup (#6210)
• K8s universe upgrade did not wait for each pod to update before moving to the next (#6360)
• Changing node count by AZ was doing a full move (#5335)
• Enabling Encryption-at-Rest without KMS configuration caused create Universe to fail silently (#6228)
• Missing SST size in the Nodes page of a universe with read replicas (#6275)
• Incorrect link in the Live queries dashboard node name (#6458)
• Failure to create a TLS-enabled universe with a custom home directory setting with on-premise provider (#6602)
• Database node health liveliness check was blocked indefinitely (#6301)

Core Database

• Critical fixes for transaction cleanup applicable to aborted transactions (observed frequently as servers reaching soft memory limit)

• Main raft fixes:

  • Incorrect tracking of flushed op id in case of aborted operations (#4150)
  • Various fixes and potential crashes on tablet bootstrap (#5003, #3759, #4983, #5215, #5224)
  • Avoid running out of threads due to FailureDetector (#5752)

• Metrics reporting could be inconsistent due to regular and intent rocksdb using the same statistics object (#5640)

• Various issues with the rocksdb snapshot mechanism used for backup/restore (#6170, #4756, #5337)

• Fixes in the YCQL API:

  • Fix parser typo when scanning binary values (#6827)
  • Omit where clause if there is no range column (#6826)
  • Fix JSONB operator execution when applying to NULL objects (#6766)
  • Fix range deletes to start and end at specified primary key bounds (#6649)

• Fixes in the YSQL API:

  • Fix crash for large DDL transaction (#6430)
  • Fixed SIGSERV in YBPreloadRelCache (#6317)
  • Fix backup-restore issues when source table was altered before (#6009, #6245, #5958)
  • Fix backup-restore failure due to unexpected NULLABLE column attribute. (#6886)

• Retry on SSL_ERROR_WANT_WRITE (#6266)

Known Issues

Yugabyte Platform

• Azure IaaS orchestration (in beta status):

  • No pricing information (#5624)
  • No support for regions with zero availability zones (AZs) (#5628)

Core Database

• Automatic Tablet Splitting:

  • While a tablet split is occurring, in-flight operations for both YCQL and YSQL APIs would currently receive errors. These would currently have to be retried at the application level currently. In the future, these will be transparently handled underneath the hood. The immediate impact for this would be that certain tools like TPCC or sysbench would fail while tablet splitting is happening.

Yugabyte Release Notes v2.5.1 Jan 14, 2021

New features

Yugabyte Platform

[#5723] Support for custom CA signed certificates for encryption in-flight
[#5556] Add alerts capability for backup tasks

Core Database

[#6010] YCQL: Cache authentication information
[#4874] YSQL: Backup for colocated databases
[#4899] YSQL: Index backfill unique indexes
[#6237] YSQL: add CREATE INDEX NONCONCURRENTLY grammar
[#5982] YSQL: Add support for CREATE TABLE/INDEX ... WITH (table_oid = x)
[#4770] YSQL: Completing feature RANGE Partitioning
Improved Clock Skew Handling
[#6370] Bump default max_clock_skew to 500ms
[#3335] Crash when too big clock skew is detected

Improvements

Yugabyte Platform

[#5888] Better error for handling when user creates concurrent backup tasks
[#6016] Pre-flight checks for create universe / edit universe / add node operations
[#4183] Add error message for invalid ssh keys for onprem providers
[#5848] Onprem provider creation failure leaves incomplete state behind
[#5652] Improve delete universe handling for failed universes
[#6254] Allow user to input multiple GFlag entries at the same time
[#4843] New Health check for Replication status in 2dc setup
[#6268] Utility script to edit universe json from command line
[#4795] Embed http proxy within yb platform to avoid need for deploying nginx
[#5799] Delete platform code related to dev/debug package installs and EPEL repo
[#6228] Enabling Encryption-at-Rest without KMS config causes Create Universe to fail silently
[#6289] Fixed degraded performance on Live Queries tab caused by huge number of DOM nodes in rows
[#6383] Disabled unused API endpoint: run_query
[#6384] Disabled unused API endpoint: run_in_shell
[#6389] Input validation for kubernetes config path
[#6386] Added input validation for access_keys API endpoint
[#6382] Added UI validation for backup target paths
#6175: Use sudo when removing prometheus snapshots during platform backup
[#6602] Fix TLS directory while provisioning YB nodes.
[#6683] Use a timeout mechanism in cluster_health.py that is more compatible with docker's lack of zombie reaping
[#6633] Platform: rename Pivotal to VMware Tanzu, add Red Hat OpenShift cloud provider

Core Database

[#6580] ycql: Log the status when statement fails for system query cache
[#6608, #6609] YCQL: Consolidate authentication error handling and messages
[YCQL][#6374] Upgrade spark-cassandra-connector version to 2.4-yb-3
#5678: [YCQL] Optimize updating non-indexed columns within indexed tables.
[#3329][YSQL] Optimized updates to reflect changes on necessary indexes only
[#5805] [YSQL] Use slots more widely in tuple mapping code and make naming more consistent
[YSQL] Clean up libpq connection code (#6481)
[#6417] [YSQL] Backport 'Skip allocating hash table in EXPLAIN-only mode'
[#6131] [YSQL] Dowgrade permission check for data directory to warning
[YSQL] Bubble up backfill error message (#6292)
[#5805] [YSQL] Don't require return slots for nodes without projection
[#5805] [YSQL] Split ExecStoreTuple into ExecStoreHeapTuple and ExecStoreBufferHeapTuple
[#5805] [YSQL] Error position support for defaults and check constraints
Improvements to 2DC x-cluster async replication
[#6169] Correctly replay write batches with external intents on bootstrap
[#6169] New format for external intents
[#6283] Fix Very Large Metric Lag for 2DC Txns
[#3522] Fix Threading Issues with CDC Consumer Writes
[#6068] Replicate Intents and Apply Messages for 2DC Txns
[#6169] Fix apply order of updated external intents records
[#4516] 2DC: Initial support for colocated databases
Improvements to Tooling
[#6589] docdb: Add option to clear placement info in yb-admin.
[#6223] master UI make hash_split more readable
[#6161] Change modify_table_placement_info to wipe read_replicas and affinitized_leaders
[#5420] ybase: Enhance YMaster admin page to display under-replicated tablets
[#1325] ybase: API for displaying YB version information
Improvements to Tablet splitting
[#4942] docdb: tablet splitting: implemented retries to post-split involved tablets for
#5937: Fixed the case when one of the tablet replicas is down during the split.
#6101 Add the flag to limit number of tablets per table
[#6424] Fix post-split compaction to be async
Improvements to core product security
[#6568] Add flag to force client certificate verification in SSL
[#6266] Replace retry counter with check that data is ready
[#6266] Retry SSL_write on SSL_ERROR_WANT_WRITE
[#6266] Fix handling SSL_write error
[#6394, #6434] docdb: Speedup system.partitions queries
[#3979] Add Transaction Cleanup to Catalog Manager Create DDLs
[#1258]: Send election request from master during table creation
[#6114] docdb: Add metrics for master YCQL system table
[#1259] Speedup DROP TABLE
[#5752] Avoid starving threadpool with run election tasks from FailureDetector
[#5755] Faster cleanup of transactions that failed to commit due to a concurrent abort
(#5996) Messenger::ScheduleOnReactor should break loop finding reactor
[#6305] Adaptive Heartbeat Reporting
[#6445] docdb: Master should rebuild YCQL system.partitions on a background thread
[#6696] Small master perf tweaks

Bug Fixes

Yugabyte Platform

[#6300] Install of s3cmd fails for default GCP OS image when airgapped
[#6085] Fixed the issue where master is brought up in read replica cluster
[#6416] Deletion with flag --node_ip fails for onprem universes.
[#6614] Fixed NPE with full move
[#6252] Fixed NPE on the metrics page
[#5942] Fixed an issue where release instance is not an option for a node should the install fail because of ssh access
[#6275] Fixed missing stats on Nodes page when universe has read replicas
[#6257] Updating user profile when smtp username or password are empty

Core Database

[#6144] YCQL: Fix handling of tablet-lookup errors in Executor::FlushAsync
[#6570] YSQL: Use IsYBRelation instead of IsYugaByteEnabled
[#6492] YSQL Avoid memcpy() with a NULL source pointer and count == 0
[YSQL] Fix IsCreateTableDone for index backfill (#6234)
[#6318, #6334] Call InitThreading in YSQL webserver process
#6317: [YSQL] Fixed SIGSERV in YBPreloadRelCache
[#6364] [YSQL] Replace CurrentMemoryContext with GetCurrentMemoryContext for all pg_extensions
#6219] [YSQL] PRIMARY KEY index in TEMP TABLE is not checked for uniqueness
(#6284) [YSQL] Clear ALTERING when there's no alter
[#5805] [YSQL] Fix run-time partition pruning for appends with multiple source rels
[#6151] [YSQL] Handle rowmark in case of read with batch of ybctids
(#6270) [YSQL] Prevent concurrent backfill index
[#6133] [YSQL] Fix procedure with an INOUT parameter in DO block
[#6061] [YSQL] ysql_dump should consistently use quotes on column names
#6009: [backup][YSQL] Fixed incorrect column-ids in restored YSQL table if original table was altered.
#5954 [YSQL] Check transaction status after read finished
#6430 [YSQL] Refresh YBCache in case postgres clears its internal cache
#6468 [YSQL] Fix read restarts of request with paging state
[#6435] docdb: fixed handling of empty bloom filter key in the write path
[#6435] Fixed bloom filter index generation for range-partitioned tablets
[#6375] Check against TabletPeer returning null tablet pointer in
(#5641) Fix for --cert_node_filename for tservers and --enable_ysql
[#6318, #6334] Call InitThreading in YSQL webserver process
[#4150] Fix incorrect tracking of flushed/synced op id in case of Raft operation abort
(#6278) Master SEGV during LB due to null TSDescriptor
[#6353] Disable rocksdb flush on all DeleteTablet calls
[#6338] Fix crash with redis workloads and snapshot restore
[#6334] Attach/detach Squeasel threads to/from libcds using callbacks
[#6245] Fixed incorrect restored table schema if the table was altered after the backup.
[#6217] Avoid a possible assertion failure in CDSAttacher destructor in Webserver
[#6170] Shutdown status resolver before destroying it in ResolveIntents
[#6482] Fix timeout handling when getting safe time in a RF1 cluster
[#6635] Fix the wrong detection of communication failure when no operations transferred
[#6678] backup: Fix restore of colocated table with table_oid already set

Known Issues

Yugabyte Platform

Azure IaaS orchestration
No pricing information provided (5624)
No support for regions with zero Availability Zones(AZs) (5628)

Core Database

Advisory on clock-skew
After commit a60a4ae00d217563cac865b3363e2c2bb8aa58ba, by default, any YB node will explicitly crash if it detects a clock skew higher than the max_clock_skew flag (default 500ms). This can be disabled by setting fail_on_out_of_range_clock_skew=false, but this could lead to consistency issues! The recommendation is that you ensure clocks are synchronized across your cluster.

Bug Fixes

Yugabyte Platform

• [#6602] Creating a TLS enabled universe with a custom home dir setting in on-prem provider fails

Core Database

• [#6394, #6434] [docdb]: Speedup system.partitions queries
• [#6435] [docdb]: fixed bloom filter index generation for range-partitioned tablets
• [#6435] [docdb]: fixed handling of empty bloom filter key in the write path
• [#6430] [YSQL] Refresh YBCache in case Postgres clears its internal cache

Known Issues

Yugabyte Platform

• [#6210] Tasks page reports an incorrect status of failed backup

New features

Yugabyte Platform

• Azure Cloud integration for Yugabyte Platform (in beta):
  Yugabyte Platform is natively integrated with Azure cloud to simplify deploying, monitoring, and managing YugabyteDB deployments. This feature automates a number of operations including orchestration of instances, secure deployments, online software upgrades, and scheduled backups, as well as monitoring and alerting. (6094, 6020)

• Yugabyte Platform operations now allow promoting a Yugabyte TServer only node to run YB-Master and YB-TServer process. (5831)

Core database

• Enhanced multi-region capabilities with geo-partitioning and follower reads
  The YugabyteDB 2.5 release adds row-level geo-partitioning capabilities as well as follower reads to the extensive set of multi-region features that YugabyteDB already had.

• Enterprise-grade security features:
  Authentication using the highly secure SCRAM-SHA-256 is now supported to limit security risks from brute force attacks and sniffing, including LDAP support for better user management and the ability to audit all database operations.

• Table-level partitions
  This allows users to split what is logically one large table into smaller sub-tables, using the following types of table partitioning schemes that PostgreSQL supports: range partitioning, list partitioning, and hash partitioning. Read more about table partitioning in YugabyteDB.

• Event triggers
  This is now supported in addition to regular table-level triggers in YSQL. While regular triggers are attached to a single table and capture only DML events, event triggers are global to a particular database and are capable of capturing DDL events. The event-based trigger framework enables detecting changes made to the data, and automating any subsequent tasks that need to be performed, which are useful in a number of use cases such as implementing a consolidated, central audit table (2379)

• Simplified cluster administration:

• Online rebuild of indexes
  This is supported for both the YSQL and YCQL APIs. This means that new indexes can be added to tables with pre-existing data while concurrent updates are happening on the cluster. The online index rebuild process creates the newly added index in the background, and transactionally enables the index once the rebuild of all the data is completed. This feature allows flexibility of adding indexes as the application needs evolve to keep queries efficient.

• Cluster execution statistics and running queries
  This can be analyzed in detail, allowing administrators to gain insights into how the database is performing. The pg_stat_statements extension, which enables tracking execution statistics of all SQL statements executed by the cluster, is supported and enabled by default. Support for pg_stat_activity has also been added, which shows information related to the activity performed by each connection. Yet another useful feature in this category is the ability to view all the live queries being executed by the cluster at any point in time.

• Detailed query plan and execution analysis
  This can now be performed with commands such as EXPLAIN and EXPLAIN ANALYZE. These commands display the execution plan generated by the planner for a given SQL statement. The execution plan shows details for any SQL statement such as how tables will be scanned (plain sequential scan, index scan), what join algorithms will be used to fetch required rows from the different tables, etc.

Improvements

Yugabyte Platform

• Enhancements to on-premises deployment workflows:
  • Do not fail universe creation if cronjobs can't be created for on-premises. (5939)
  • Remove pre-provision script requirement for airgapped installations. (5929)
  • "Sudo passwordless" in on-premises cloud provider configuration toggle is renamed.
  • Allow yugabyte user to belong to other user groups in Linux. (5943)
  • Added a new "Advanced" section in on-prem cloud provider configuration which includes:
    • Use hostnames
    • Desired home directory
    • Node exporter settings
  • Improvements to installation of Prometheus Node Exporter utility workflow. (5926)
    • The node exporter option is now available in the cloud configuration under advanced settings.
    • Supports bringing your own node exporter user.
• Make YEDIS API optional for new Universes and no change in behavior of existing universes. (5207)
• UI/UX improvements for YB Platform:
  • Add visual feedback when backup or restore is initiated from modal. (5908)
  • Minor fixes to primary cluster widget text issue. (5988)
  • Show pre-provision script in UI for non-passwordless sudo on-premises provider. (5550)
  • Update backup target and backup pages. (5917)
• For YugabyteDB universes with replication factor (RF) > 3, change the default min_num replicas for even distribution of AZs across all regions. (5426)
• Added functionality to create IPv6-enabled universe in Kubernetes. (5309, 5235)

Core database

• Support for SQL/JSON Path Language (jsonb_path_query). (5408)
• Remove range-loop-analysis and fallthrough errors in our code. (5812)
• Incorrect index update if used expression result is still unchanged. (5898)
• As part of the tablet splitting feature:
  • Implemented cleanup of the tablet for which all replicas have been split for. (4929)
  • Compaction improvements (5523)
• Improve performance for sequences by using higher cache value by default. Controlled by a YB-TServer configuration flag --ysql_sequence_cache_minval. (6041)
• Added compatibility mode in the yb_backup.py script for YugabyteDB version earlier than v2.1.4. (5810)
• Stability improvements to core database - make exponential backoff on lagging RAFT followers send NOOP. (5527)
• Added use of separate metrics objects for RegularDB and IntentsDB. (5640)
• Improvements to 2DC (xCluster replication) by adding TLS support for cleaning up cdc_state table. (5905)

Bug fixes

Yugabyte Platform

• Fix for Universe disk usage shows up empty on the universe page. (5548)
• Fix on on-premises backup failures due to file owned by the root user. (6062)
• Fix for a bug where user operation to perform a change to nodes count by AZ was doing a full move. (5335)
• Fixes for Yugabyte Platform data backup script for Replicated-based installations
• Fixes to Client Certificate start time to use UTC during download. (6118)
• Fixes for migration if no access keys exist yet. (6099)
• Fix to resolve issues caused by starting a YB-TServer node when another YB-Master node is down in the universe. (5739)
• Use the correct disk mount point while calculating disk usage of logs. (5983)

Core database

• Fix for bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index columns. (5811)
• Improve fault tolerance of DDLs and reduce version mismatch errors in YSQL. (3979, 4360)
• Fixes to incorrect column-ids in the restored table if the original table was altered. (5958)
• Fixes timeout bug in Yugabyte Platform when there are read replicas. This fix will ignore read replica YB-TServers when running AreLeadersOnPreferredOnly. (6081)
• Fixes to restore of YSQL Backups after dropping and recreating a database. (5651)
• Fixes to a bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index columns. This is fixed by setting the read time for backfill write requests to be the safe time chosen for backfill read requests rather than the hard-coded time 50. (5811)
• Fixes to delete backups for TLS-enabled universes. (5980)

Known Issues

Yugabyte Platform

• Azure IaaS orchestration:
  • No pricing information provided. (5624)
  • No support for regions with zero availability zones (AZs). (5628)

Yugabyte Release Notes v2.2.5 Dec 2, 2020

Bug Fixes

Yugabyte Platform

• Fix for updating user profile when SMTP username or password is empty (6257)
• Fix for editing YugabyteDB Universe placement info logic (6250)
• Fixes to delete unnecessary Prometheus snapshot when backing up YB Platform data (6175)
• Fix for an issue where Client Certificates are not valid on download due to incorrect validity time (6118)
• Safeguard extra migration for pre-provisioning (6181)

Core Database

• Fixes to the logic for SSL write to continue retrying, as long as SSL_write produces new data to send (6266)
• Fix TServer crash with YEDIS workloads and snapshot restore (6338)
• Disable Rocksdb flush on all DeleteTablet calls (6353)
• Shutdown status resolver before destroying it in ResolveIntents (6170)

Known Issues

Yugabyte Platform

• Tasks page reports an incorrect status of failed backup (6210)

Platform Support Package Improvements (Only applicable to Non-Replicated installations)

Yugabyte Platform

• It is required to pull the latest platform-support-packages.tar.gz binaries for the fix related to issue (6175) - “Fixes to delete unnecessary Prometheus snapshot when backing up YB Platform data”

Yugabyte Release Notes v2.2.4 Oct 30, 2020

New features

Yugabyte Platform

Yugabyte Platform operations now allow promoting a Yugabyte TServer only node to run Yugabyte Master and TServer process (5831)

Improvements

Yugabyte Platform

Enhancements to on-prem Cloud Provider configuration
When creating onprem universes, YW will automatically run provisioning if airgap is enabled now, instead of asking the users to run the pre-provision script. Users will need to run the script only when creating universes without passwordless sudo access.
Do not fail universe creation if cronjobs can't be created for on-prem (5939)
Remove pre-provision script requirement for air-gapped installations (5929)
"Sudo passwordless" in on-prem cloud provider configuration toggle is renamed to “Manually Provision Node”
Added a new "Advanced" section in on-prem cloud provider configuration which includes
Use hostnames
Desired home directory
Node exporter settings
Pre-provisioning of ‘yugabyte’ user is supported for on-prem Cloud Provider. (YB devops script won't run create_user if yugabyte user already exists) (6086)
Improvements to installation of Prometheus Node Exporter utility workflow (5926)
Prometheus Node exporter option is now available in the cloud configuration under advanced settings
Supports bringing your own node exporter user
UI/UX improvements for YB Platform
Add visual feedback when backup or restore is initiated from modal (5908)
Add new button for downloading root cert in Certificates page (5521)
Use the correct disk mount while calculating the disk usage of logs (5983)
Core Database
YSQL Support ROWS_PER_TRANSACTION option for COPY FROM stdin (6069)
Bug Fixes
Yugabyte Platform

[Replicated] Fixed an issue that would leave Replicated in a bad state if the network connection drops while uploading an airgap bundle, preventing the installation from proceeding. (https://release-notes.replicated.com/release-notes/2.49.0/)
Fix for the check if changing node count by AZ affects master node placement and requires a full move (5335)
Fix delete backup failure for TLS enabled universes (5980)
Universe disk usage shows up empty on the universe page (5548)
Fix for starting a YB TServer Node when another YB Master is down causes issues (5739)
Fix platform backup script for replicated installations (6130)

Core Database
[docdb] Fix a rare bug on restore which could clear more metadata than necessary (5788)
[docdb] Fix a rare race condition between transactions and snapshots that could lead to a deadlock (6170)

[YSQL] Fix an issue with setting bind address for YSQL metrics endpoint for IPv6 scenarios (4901)
[YCQL] Fix for bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index column(s) (5811)
[YCQL] Fix a bug with update and expression indexes, where an index row would get deleted if the column is updated without changing the indexed value (expression) (5898)
[YSQL] Fix OOM with \copy or COPY FROM stdin for large files (5603)
Known Issues
Yugabyte Platform
Client Certificates are not valid on download due to incorrect validity time (6118)
Tasks page reports incorrect status of failed backup (6210)

Platform Support Package Improvements
(Only applicable to Non-Replicated installations)
Yugabyte Platform
Fix for On-prem YB Platform backup fails due to file owned by root (6062)

New features

Yugabyte Platform

• Introducing beta support for Azure IaaS orchestration in Yugabyte Platform cloud provider configuration. (6094, 6020)
• Yugabyte Platform operations now allow promoting a Yugabyte TServer only node to run YB-Master and YB-TServer process. (5831)

Core database

• [YSQL] Introducing support for Event Triggers that are global to a particular database and are capable of capturing DDL events. (2379)

Improvements

Yugabyte Platform

• Enhancements to on-premises deployment workflows:
  • Do not fail universe creation if cronjobs can't be created for on-premises. (5939)
  • Remove pre-provision script requirement for airgapped installations. (5929)
  • "Sudo passwordless" in on-premises cloud provider configuration toggle is renamed.
  • Allow yugabyte user to belong to other user groups in Linux. (5943)
  • Added a new "Advanced" section in on-prem cloud provider configuration which includes:
    • Use hostnames
    • Desired home directory
    • Node exporter settings
  • Improvements to installation of Prometheus Node Exporter utility workflow. (5926)
    • The node exporter option is now available in the cloud configuration under advanced settings.
    • Supports bringing your own node exporter user.
• Make YEDIS API optional for new Universes and no change in behavior of existing universes. (5207)
• UI/UX improvements for YB Platform:
  • Add visual feedback when backup or restore is initiated from modal. (5908)
  • Minor fixes to primary cluster widget text issue. (5988)
  • Show pre-provision script in UI for non-passwordless sudo on-premises provider. (5550)
  • Update backup target and backup pages. (5917)
• For YugabyteDB universes with replication factor (RF) > 3, change the default min_num replicas for even distribution of AZs across all regions. (5426)
• Added functionality to create IPv6-enabled universe in Kubernetes. (5309, 5235)

Core database

• Support for SQL/JSON Path Language (jsonb_path_query). (5408)
• Remove range-loop-analysis and fallthrough errors in our code. (5812)
• Incorrect index update if used expression result is still unchanged. (5898)
• Various improvements to RaftConsensus tests. (4548, 5912)
• As part of the tablet splitting feature:
  • Implemented cleanup of the tablet for which all replicas have been split for. (4929)
  • Compaction improvements (5523)
• Improve performance for sequences by using higher cache value by default. Controlled by a YB-TServer configuration flag --ysql_sequence_cache_minval. (6041)
• Added compatibility mode in the yb_backup.py script for YugabyteDB version earlier than v2.1.4. (5810)
• Stability improvements to core database - make exponential backoff on lagging RAFT followers send NOOP. (5527)
• Added use of separate metrics objects for RegularDB and IntentsDB. (5640)
• Improvements to 2DC (xCluster replication) by adding TLS support for cleaning up cdc_state table. (5905)

Bug fixes

Yugabyte Platform

• Fix for Universe disk usage shows up empty on the universe page. (5548)
• Fix on on-premises backup failures due to file owned by the root user. (6062)
• Fix for a bug where user operation to perform a change to nodes count by AZ was doing a full move. (5335)
• Fixes for Yugabyte Platform data backup script for Replicated-based installations
• Fixes to Client Certificate start time to use UTC during download. (6118)
• Fixes for migration if no access keys exist yet. (6099)
• Fix to resolve issues caused by starting a YB-TServer node when another YB-Master node is down in the universe. (5739)
• Use the correct disk mount point while calculating disk usage of logs. (5983)
• Fixes to delete backups for TLS-enabled universes. (5980)

Core database

• Fix for bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index columns. (5811)
• Improve fault tolerance of DDLs and reduce version mismatch errors in YSQL. (3979, 4360)
• Fixes to incorrect column-ids in the restored table if the original table was altered. (5958)
• Fixes timeout bug in Yugabyte Platform when there are read replicas. This fix will ignore read replica YB-TServers when running AreLeadersOnPreferredOnly. (6081)
• Fixes to restore of YSQL Backups after dropping and recreating a database. (5651)
• Fixes to a bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index columns. This is fixed by setting the read time for backfill write requests to be the safe time chosen for backfill read requests rather than the hard-coded time 50. (5811)

Known Issues

Yugabyte Platform

• Azure IaaS orchestration:
  • No pricing information provided. (5624)
  • No support for regions with zero availability zones (AZs). (5628)

New features

Yugabyte Platform

• Enhancements to RBAC with a new “Backup Admin” role which allows all read-only permissions plus back up operations #5311
• Display table sizes in the Tables page #3581
• On-prem Cloud provider instances table list can be sorted and filtered #4757
• API for disabling auto-generation of database nodes certificate #5545
• Backup retention policy for auto-deletion of older backups #4493

Core database

• [YCQL] Introducing Audit Logging feature for YCQL operations #5887
• [YSQL] Support for ALTER COLUMN type that does not require on-disk changes #4424
• [YSQL] Support for Transactional batch size in COPY FROM command syntax #2855, #5453
• [YCQL] Finer grained permission control for TRUNCATE #5443
• [YSQL] Enable 2DC replication bootstrap for YSQL Tables #5601

Improvements

Yugabyte Platform

• Backup improvements
  • YCQL, YSQL, and YEDIS backups are split into different tabs on the backup UI screen
  • Back up operation creates a single restore object regardless of “transactional” field selection
  • Added a new field parallel thread for multi-threading backups
  • YSQL backups only support namespace level backup
  • Backup table list has been revamped with the new UX and shows more details - expiration time, duration, backup ty
  • Universe can be restored with a single action
  • Backup objects can be manually deleted
  • Scheduled backups cron expression shows the next run time in local browser time and the label displays that cron supports UTC only #4709
• Improve the Replication graph for xDC replication set up to display the graph and metric names cleanly #5429
• UI improvements for displaying On-prem instances
  • Instance ID is now optional and also shown in the nodes page #4760
  • “In Use” column has been renamed to “Universe Name”
  • Supports adding multiple instances in different rows instead of comma-separated entries
• Supports generating API tokens for SSO-enabled users
• Yugabyte Platform restart cancels any tasks that were in progress
• Remove sudo requirement in DB backup script #5440

Core database

• [DocDB] Removal of unreachable (dead) nodes from YugabyteDB UI #4759
• [DocDB] Improvements with selecting better default flags for Index backfill #5494
• [DocDB] Lower the leader load balancer parallelism for performance improvements #5461

Bug fixes

Yugabyte Platform

• [Platform] Security hardening: Added CSRF token support
• [Platform] Support for PingFederate OIDC SSO login for Non Replicated deployments
• [Platform] Allow on-prem nodes to be reused after releasing them from a universe #5703

Core Database

• [YCQL] Fix for CQL Index scans when ORDER BY columns exist in the table but not in chosen index #5690
• [YSQL] Avoids Pushdown down of UPDATE with RETURNING clause #5366
• [YSQL] Fix for UPDATE operation with partial and expression Indexes #4939
• [YSQL] OOM issue fix in COPY FROM query #2855
• [YSQL] OOM issue fix in COPY TO query #5205
• [DocDB] Crash fix for DNS-enabled Yugabyte Universes #5561
• [DocDB] Fix for in-memory state not getting updated correctly when using read-from-follower query semantics #1052
• [DocDB] For Yugabyte Universes using cross-cluster async replication (2DC), added RPC throttling on idle CDC tables to avoid wasting CPU #5472
• [DocDB] Fix for rare deadlock scenario when the node switches from Leader to Follower state of the tablet and there are pending operations on Leader side #5741
• [DocDB] Fix to ensure all Yugabyte Tablet Servers are accounted for when Master fails over #5501

Known Issues

Yugabyte Platform

• Universe disk usage shows up empty on the universe page #5548

Platform Support Package Improvements (Only applicable to Non-Replicated installations)

Yugabyte Platform

• Platform backup script needs to preserve attributes on copy from Prometheus snapshot #5612
• Platform backup fails if /opt/yugabyte/release directory does not exist #5615
• Platform backup script needs to stop Prometheus service during the restore of Prometheus data #5685

Note:

Prior to v2.0, YSQL was still in beta. Upon the release of v2.0, a backward-incompatible file format change was made for YSQL. For existing clusters running pre-2.0 release with YSQL enabled, you cannot upgrade to v2.0 or later. Instead, export your data from existing clusters and then import the data into a new cluster (v2.0 or later).

YSQL

• Enable pg_stat_statements extension by default. #5750
• Clean up unused PgStatement objects on hot path after usage to avoid memory consumption. #5374
• Fix restarting writes (INSERT and UPDATE) on transaction conflicts and avoid surfacing transaction errors to user applications. #4291
• Enable batching of updates for YSQL procedures and functions. #5257

YCQL

• Add additional system queries (SELECT_LOCAL, SELECT_LOCAL_TOKENS, and SELECT_PEERS_TOKENS) for caching. #5753
• Fix crash for YCQL index scan when ORDER BY columns exist in table, but not in the chosen index, and raise an error. #5690

Core database

• Allow setting a custom replacement policy per table through yb_admin. #5368
• Do not clean up transaction that potentially was committed. #5789 #5783 #5755
• Fix CDC shutdown races. Moved thread pool shutdown before other CDC objects cleanup. Added explicit CDC poller clear in CDC consumer shutdown. #5583
• Fix preparer deadlock after switching from leader to follower. #5741
• Fix RaftGroupMetadata construction issues found using PVS Studio. #5578
• Change rlimit flags to have at-least semantics. Only change the process soft limit if it is smaller than the limit specified by the corresponding flag value. #5715
• Add extra logging when initializing syscatalog. #5708
• Set default webserver_interface to rpc_bind_address IP/host component. #5320
• Add environmental variables functionality to yugabyted. New env variables for YSQL (YSQL_USER, YSQL_PASSWORD, and YSQL_DB) and YCQL(YCQL_USER, YCQL_PASSWORD, and YCQL_KEYSPACE). (#5499
• Improve cleanup of aborted transactions. #5669
• Fix dumping an empty batch by updating MaxReplicateOpId. #5679
• Set default staleness for read from followers (using max_stale_read_time_ms) to 60 seconds. Previously, a follower could return requested data no matter how state the data is. #5590
• Calculate the placement UUID from Raft upon tserver registration. #5558
• Enable ignoring deleted tablets. #5122
• Remove unreachable nodes from UI (/tablet-servers page) after nodes are removed from the cluster. #4759

Yugabyte Platform

• Fix issue parsing multiple comma-separated addresses for rpc_bind_addresses. #5634
• Add functionality to delete old backups after a specified time period. In the Create Backup dialog, added Number of Days to Retain Backup field. #4493
• Combine the Platform and community Grafana dashboards. Grafana dashboard shows the node status and other panels correctly. #5707
• Sum table sizes across all nodes of a universe. #5745
• Fixes releasing OnPrem nodes should not delete them from the universe metadata. Fixes releasing a node should only attempt to shut down processes on that node. Nodes can be re-added using Node > Actions > Add flow. #5703 #5791 #5792
• Add additional property to MetricsPanel to handle width of Replication graph and increase height. Aggregates replication metrics values from all tservers into an average and max value graph across all tservers. #5429 #5613 #5632
• Update yb-client JAR version to 0.8.1. #5759
• Stop and restart relevant services during Yugabyte Platform backup. Backups stop and restart services. Restores stop and restart Yugabyte Platform and Prometheus, if necessary. #5685
• Authenticate API tokens even when using OAuth. This allows curl requests. #5587
• Fix Metrics tab and columns misalignment on table page. #5582
• Rename In Use column to Universe Name and show NOT USED if the node is not in use. #5713
• Do not generate node certificates for custom certificates. #5545
• Make instance name field optional for on-premises provider. #5234
• Fix button for Metrics tab to reset active and focus state when key is unpressed. #5620
• Add Docker- and VM-aware options to the backup script and will include Prometheus snapshots. #5120 #5615 #5612 #5614

Note:
Prior to v2.0, YSQL was still in beta. Upon release of v2.0, a backward-incompatible file format change was made for YSQL. For existing clusters running pre-2.0 release with YSQL enabled, you cannot upgrade to v2.0 or later. Instead, export your data from existing clusters and then import the data into a new cluster (v2.0 or later).

Features and changes

YSQL

• Fixes OOM when using COPY <table> FROM <file> to load data from a large file to a table. #5453
• Enable 2DC replication bootstrap for YSQL tables. This allows the producer to know where to start replicating from. #5601
• Fix CREATE TABLE is 4-5x slower using Docker on Mac than not using Docker. Speeds up table creation by buffering writes to postgres system tables, caching pinned objects, and significantly reducing write RPC calls. #3503
• Roll back the catalog version made in commit 46f3701 so that 2.3 upgrades can proceed. #5408

Core database

• Quickly evict known unresponsive tablet servers from the tablet location cache. Applies only to follower reads. For example, when tablet servers are not replying to RPC calls or dead — not sending heartbeats to the master for 5 minutes. This could also happen after decommissioning nodes. #1052

Yugabyte Platform

• Add search input and data sorting to the on-premises instances table list. Click arrows next to column titles to sort. Use the Search form to search multiple columns. #4757
• Add the ability to change the user role (Admin, ReadOnly, or BackupAdmin) from the UI by an admin. Also, fix stale users list after creation or deletion of a user and disable Save buttons at Customer Profile tabs for ReadOnly users. #5311
• Before deleting a universe, stop the master/tserver processes and release the instance so that if started again, the processes are not still running. #4953

Note:

Prior to version 2.0, YSQL was still in beta. As a result, the 2.0 release included a backward-incompatible file format change for YSQL. If you have an existing cluster running releases earlier than 2.0 with YSQL enabled, then you will not be able to upgrade to version 2.0+. Export from your existing cluster and then import into a new cluster (v2.0 or later) to use existing data.

YSQL

• Fix OOM when running large COPY TO statements by creating a new memory context for the loop over retrieved rows and resetting it after processing each row. #5205
• Fix OOM on file-sourced YB relations in the COPY FROM statement. Reset memory context regularly, per row, when memory is resettable and when rows are read from a file (not stdin). #5561
• Support transactional batch size for COPY FROM command with OOM fix. Batch sizes can be passed in with ROWS_PER_TRANSACTION in the COPY OPTION syntax. #2855 #5453
• For index backfill flags, use better default values. Set index_backfill_rpc_timeout_ms default from 60000 to 30000 and change backfill_index_timeout_grace_margin_ms default from 50 to 500. #5494
• Remove spurious error message "0A000: Alter table is not yet supported" from CREATE OR REPLACE VIEW. #5071
• Prevent consistency violations when a partitioned table has foreign key constraints due to erroneous classification as a single-row transaction. #5387
• Fix restore from a distributed backup fails for tables using SPLIT INTO without a primary key. #4993
• Fix the wrong result by avoiding pushdown of UPDATE statement with the RETURNING clause. #5366
• Improve error message when UPDATE changes partition so its clear what went wrong, fixed yb_pg_foreign_key pg_regress test for semantic merge conflict when updating primary keys and row-level partitioning. #659 #5179 #5310
• Allow the UPDATE statement to change primary key columns. #659
• Support row-level partitioning. #5179
• Block usage of TABLEGROUP with the SPLIT clause. For CREATE TABLE, usage is blocked in the grammar. For CREATE INDEX, if NO TABLEGROUP was provided, then presplitting for the index is allowed; otherwise, an error is issued. #5352
• Buffered operations may share single RPC with read operation. Reducing the number of RPC calls speeds up the CREATE TABLE statement. For example, the total number of RPC calls is dropped from 66 to 58 for a simple table like CREATE TABLE t(k INT PRIMARY KEY). #5177
• Add new ysqlsh describe metacommands for tablegroups: \dgr[+] [grpname] to describe tablegroups, \dgrt[+] [grpname] lists all tables/indexes within the specified tablegroup (or within all tablegroups if grpname is not specified), and \d <table_name> is modified to include tablegroup information in the footer, if any. #5088
• Add ALTER TABLEGROUP statements to support ALTER TABLEGROUP tablegroup_name RENAME TO ... and ALTER TABLEGROUP tablegroup_name OWNER TO .... Also changes pg_tablegroup entry corresponding to tablegroup_name to properly reflect new grpname and grpowner (if the user has proper permissions or ownership to issue the ALTER TABLEGROUP statement. #5249
• Add support for indexes to opt out of tablegroups (NO TABLEGROUPS) or select their own tablegroup (TABLEGROUP group_name). #5293
• For ysql_dump, enable serializable-deferrable mode by default. Add new --no-serializable-deferrable flag to disable the default mode. #5128
• Support ALTER COLUMN type for variants that don't require on-disk changes. Specifically only allowing ALTER COLUMN type for varch(n) and varbit(n). For example, changing column type from varchar(50) to varchar(255). #4424
• Enable USING clause in DELETE statement and FROM clause in UPDATE statement. #738 #5262
• Support GRANT/REVOKE/ALTER DEFAULT PRIVILEGES for tablegroups. #5087 #5160
• Implement tablegroup query layer changes. #4525
• Replace "Foreign Scan" with "Seq Scan" for EXPLAIN statement. #2076
• Fix initdb when index backfill is enabled. #5027
• Properly handle empty delete with backfill by sending appropriate messages to the client, including setting it skipped and rows_affected_count to 0. #5015
• DROP INDEX statement invalidates table cache entry for the index table but should invalidate the table cache entry for the indexed table. #4974
• yb-admin create_database_snapshot command should not require ysql. prefix for database name. #4991
• For non-prepared statements, optimize pg_statistic system table lookups and update debugging utilities. #5051
• Correctly show beta feature warnings by default. #5322

YCQL

• For WHERE clause in CREATE INDEX statement, return a Not supported error. #5363
• Fix TSAN issue in partition-aware policy for C++ driver 2.9.0-yb-8 (yugabyte/cassandra-cpp-driver). #1837
• Support YCQL backup for indexes based on JSON-attribute. #5198
• Correctly set release_version for system.peers queries. #5407
• Reject TRUNCATE operations when ycql_require_drop_privs_for_truncate flag is enabled. When enabled, DROP TABLE permission is required to truncate a table. Default is false. #5443
• Fix missing return statement in error case of the SetPagingState method in statement_params.cc. #5441
• Enable backfilling of transactional tables by default. #4708
• Fix ycqlsh should return failure when known that the create (unique) index has failed. #5161

Core database

• Fix core dump related to DNS resolution from cache for Kubernetes universes. #5561
• Fix yb-master fails to restart after errors on the first run. #5276
• Show better error message when using yugabyted and yb-master fails to start. #5304
• Disable ignoring deleted tablets on load by default. #5122
• [CDC] Improve CDC idle throttling logic to reduce high CPU utilization in clusters without workloads running. #5472
• For server_broadcast_addresses flag, provide default port if not specified. #2540
• [CDC] Fix CDC TSAN destructor warning. #4258
• Add API endpoint to download the root certificate file. #4957
• Do not load deleted tables and tablets into memory on startup. #5122
• Set the follower lag for leaders to 0 by resetting timestamp to the maximum value when a peer becomes a leader and when the peer loses leadership. #5502
• Flow keyspace information from yb-master to yb-tserver. #3020
• Implement meta cache lookups throttling to reduce unnecessary thrashing. #5434
• Fix rpcz/statements links in yb-tserver Web UI when pgsql_proxy_bind_address and cql_proxy_bind_address are 0.0.0.0. #4963
• DumpReplayStateToStrings should handle too many WAL entries scenario. Also, log lines only display fields critical for debugging and do not show customer-sensitive information. #5345
• Find the difference between the replica map and consensus state more quickly using map lookup. #5435
• Improve failover to a new master leader in the case of a network partition or a dead yb-tserver by registering tablet servers present in Raft quorums if they don't register themselves. Also, mark replicas that are in BOOTSTRAPPING/NOT_STARTED state with its true consensus information instead of marking it as a NON_PARTICIPANT. #4691
• Add YB-Master UI changes to support tablegroups and colocation. #5086
  • Adds a column to the user and index tables to display the YSQL OID information about the colocation parent table (if any).
  • For tablegroups, it displays the OID of the table's tablegroup (same as the OID present in pg_tablegroup as well as the reltablegroup column of pg_class for that relation).
  • For colocated databases, it displays the OID of the database.
  • Also, creates a table to display information about the parent tables. Wrap the names and uuid of the parent tables. Display YSQL OIDs as explained above.
• Fix MetaCache::TAbleData::stale is not getting reset back to false. #5245
• Do not crash yb-tserver when the op_id of the last WAL file is less than the committed op_id. #1560
• Add CREATE TABLEGROUP and DROP TABLEGROUP flow. #4525
• Add yb-admin list_snapshots SHOW_DELETED flag to show deleted snapshots that are still retained in memory. #5332
• Move read path functions to a separate file. #4944
• Wait for yb_client_admin_operation_timeout_sec for table creation in yb-admin import_snapshot task. #5295
• Create a cache for system-wide queries. #5043
• Annotate access to enable_collect_cdc_metrics flag and prevent TSAN warnings. #5303
• Raise yb-master and yb-tserver process ulimits to hard limits when appropriate. #4943
• Add a pre-flush step, triggered by yb-master, to snapshot operations to minimize the amount of data being flushed in sync. #4718
• Use Strand to remove intents and avoid wasting threads. #5265
• Add metric severity level [PORT] and server-side level-based filtering for metrics and prometheus-metrics. #4686
• Decrease logging frequency for no local transaction status tablet from on every warning to once per second. #5035
• Fix MonoTime subtraction implementation to return result of the operation instead of lhs. #5020
• When dropping tables, delete snapshot directories for deleted tables. #4756
• Set up a global leader balance threshold while allowing progress across tables. Add load_balancer_max_concurrent_moves_per_table and load_balancer_max_concurrent_moves to improve performance of leader moves. And properly update state for leader stepdowns to prevent check failures. #5021 #5181
• Reduce default value of load_balancer_max_concurrent_moves from 10 to 2. #5461
• Improve bootstrap logic for resolving transaction statuses. #5215
• Avoid duplicate DNS lookup requests while handling system.partitions table requests. #5225
• Handle write operation failures during tablet bootstrap. #5224
• Ensure strict_capacity_limit is set when SetCapacity is called to prevent ASAN failures. #5222
• Drop index upon failed backfill. For YCQL, generate an error to CREATE INDEX call if the index backfill fails before the call is completed. #5144 #5161
• Allow overflow of single-touch cache if the multi-touch cache is not consuming space. #4495
• Don't flush RocksDB before restoration. #5184
• Fix yugabyted fails to start UI due to class binding failure. #5069
• On yb-tserver restart, prioritize bootstrapping transaction status tablets. #4926
• For yb-admin commands, clarify syntax for namespace. #5482
• Improve for-loops to avoid unnecessary copies and remove range-loop-analysis warnings. #5458
• Apply a large transaction in a separate thread if the transaction will not fit into the limit of key-value pairs. The running transaction is removed from memory after all its intents are applied and removed. #1923
• Fix SEGV in Master UI when registering YB-TServer from Raft. #5501
• Heartbeats process only new consensus information unless master_ignore_stale_cstate flag is disabled. #5301
• Remove KernelStackWatchdog and use LongOperationTracker to waste fewer resources per thread. #5226
• Change default value of metric_node_name flag from DEFAULT_NODE_NAME to hostname:port. #4859

Yugabyte Platform

• For S3 backups, install s3cmd required for encrypted backup and restore flows. #5593
• When creating an on-premises provider, remove YB_HOME_DIR if not set. #5592
• Update to use templatized values for setting TLS flags in Helm Charts. #5424
• For YCQL client connectivity, allow downloading root certificate and key from the Certificates menu. #4957
• Bypass CSRF checks when registering and logging in due to bug with CSRF token not being set. #5533
• Set node certificate field to subject of CA instead of the issuer of CA. #5377
• Add option in the universe creation form to control whether node_exporter is installed on provisioned nodes (including creating prometheus user + group). #5421
• Add Microsoft Azure provider UI and related changes to Create Universe form. #5378
• Create default network resources for multi-region deployments in Microsoft Azure if they don't exist and store for use during universe creation. #5388
• For Microsoft Azure, use preexisting network resources. #5389
• Set Assign Public IP correctly using the UI. #5463
• Fix AreLeadersOnPreferredOnly times out when setting preferred zones in Edit Universe. #5406
• Fix yb_backup.py script to remove the sudo requirement and correctly change the user. #5440
• Add CSRF tokens to forms. #5419
• Allow configuration of OIDC scope (for SSO use) as well as email attribute field. #5465
• If Username and Password are left empty in the Custom SMTP Configuration, remove smtpUsername and smtpPassword from the payload. #5439
• Add UTC label to indicate cron expression must be relative to UTC. Also, add help text indicating when the next scheduled job will run. #4709
• Move the source of truth for communication ports to the universe level. #5353
• Use the appropriate range for the time period (over the entire step window rather than the last minute) when querying Prometheus for metrics. #4203
• Support backing up encrypted at rest universes. #3118
• Only display "Rolling Upgrade Delay" if the upgrade type is "Rolling". #5362
• Add guarding against never-ending loop when running yb_backup.py script and yb-server doesn't have any data directories. #5358
• Fix metric panel displaying incorrect time interval when reloading after changing graph filter. #5294
• Fix logging on health check script failure. #5340
• Rename label for enabling server-side encryption in S3 backups to Encrypt Backup for both backups and scheduled backup modals. #5273
• Add a VM backup script that includes Prometheus backups. #5120
• Add BackupAdmin role to authorize users to create a backup and restore tasks on universes, but not given other administrative privileges. #4694
• Modify yb_backup script to add "round-robin" behavior to thread allocations to better spread loads across nodes in parallel during snapshot upload or download. #4987
• Fix invalid yb_backup.py script behavior if --keyspace flag is not specified. #5199
• Add subtasks to check the number of pods deployed during upgrades. #5174
• Make use of parallelism argument for yb_backup.py script. Explicitly pass the number of threads to concurrently run backup script through UI. The form defaults to the backup script default value of 8 threads if no value is provided. #5283
• The Remove Node action should be available if the Add Node action fails. #5123
• Add the ability for user to specify the S3 backup target host that their bucket is located on. #3684
• Check if the on-premise node instance has a universe that is using it and make a link to the Universe Overview page. #4758
• Fix Cannot read property 'displayName' of undefined error. #5267
• Use Host Names setting is not displaying correctly in Provider Configuration page. #5252
• Disable glob before running the cleanup of old log files. #5169
• Schedule job for setUniverseKey should handle errors on a per-customer basis. #3142
• After the Yugabyte Platform (Yugaware) restart, any in-progress tasks should be canceled and marked as failed. #5045
• Redirect from login back to index if the session is already validated. Logout should redirect to the login endpoint. #5019
• Fix updating of Universes list after creating a new universe. #4784
• Fix restore payload when renaming table to include keyspace. And check for keyspace if tableName is defined and return invalid request code when keyspace is missing. #5178
• Add the ability to override communication port default values during universe creation. #5354

Note:

Prior to version 2.0, YSQL was still in beta. As a result, the 2.0 release included a backward-incompatible file format change for YSQL. If you have an existing cluster running releases earlier than 2.0 with YSQL enabled, then you will not be able to upgrade to version 2.0+. Export from your existing cluster and then import it into a new cluster (v2.0 or later) to use existing data.

YSQL

• Fix failed backup if restored table was deleted before restoration. #5274
• Newly elected YB-Master leader should pause before initiating load balancing. #5221
• Fix backfilling to better handle large indexed table tablets. #5031
• Fix DROP DATABASE statement should work with databases deleted on YB-Master. #4710
• For non-prepared statements, optimize pg_statistic system table lookups. #5051
• [CDC] Avoid periodic querying of the cdc_state table for xDC metrics if there are no replication streams enabled. #5173

YCQL

• Implement DNS cache to significantly reduce CPU loads due to a large number of DNS resolution requests (especially for YCQL connections). Adds dns_cache_expiration_ms flag (default is 1 minute). #5201
• Fixed incorrect names de-mangling in index creation from CatalogManager::ImportSnapshot(). #5157
• Fixed crashes when inserting literals containing newline characters. #5270
• Reuse CQL parser between processors to improve memory usage. Add a new cql_processors_limit flag to control processor allocation. #5057

Core database

• Fix yugabyted fails to start UI due to class binding failure. #5069
• Show hostnames in YB-Master and YB-TServer Admin UI when hostnames are specified in --webserver_interface, --rpc_bind_addresses, and --server_broadcast_addresses flags. #5002
• Skip tablets without intents during commits, for example, the case of an update of a non-existing row. #5321
• Fix log spew when applying an unknown transaction and release a mutex as soon as possible when the transaction is not found. #5315
• Fix snapshots cleanup when snapshots removed before restart. #5337
• Replace SCHECK with LOG(DFATAL) when checking for restart-safe timestamps in WAL entries. #5314
• Replace all CHECK in the load balancer code with SCHECK or RETURN_NOT_OK. #5182
• Implement DNS cache to significantly reduce CPU loads due to a large number of DNS resolution requests (especially for YCQL connections). Adds dns_cache_expiration_ms flag (default is 1 minute). #5201
• Avoid duplicate DNS requests when handling system.partitions table requests. #5225
• Fix leader load balancing can cause CHECK failures if stepdown task is pending on the next run. Sets the global leader balance threshold while allowing progress to be made across tables. Adds new load_balancer_max_concurrent_moves_per_table flag to limit number of leader moves per table. #5181 and #5021
• Set the async YBClient initialization future in TabletPeer constructor to ensure tablet bootstrap logic can resolve transaction statuses. #5215
• Handle write operation failures during tablet bootstrap. #5224
• Drop index upon failed backfill. #5144 #5161
• Fix WAL overwriting by new leader and replay of incorrect entries on tablet bootstrap. #5003 #3759 #4983
• Avoid taking a unique lock when starting a lookup request. #5059
• Set 2DC lag metrics to 0 if not the leader and if the replication is deleted. #5113
• Set the table to ALTERING state when fully_* is populated. #5139
• Not the leader errors should not cause a replica to be marked as failed. #5072
• Use the difference between follower's hybrid time and its safe time as a measure of staleness. #4868

Yugabyte Platform

• Add Master section below Tablet Server section in Metrics page. #5233
• Add rpc_connections_alive metrics for YSQL and YCQL APIs. #5223
• Fix restore payload when renaming table to include keyspace. Disable keyspace field when restoring universe backup. #5178
• Pass in ssh_user to air-gap provision script and add to the on-premise template. #5132
• Add the --recursive flag to AZCopy for multi-table restore. #5163
• Fix transactional backup with specified tables list by including keyspace in payload. #5149
• Fix undefine object property error when Prometheus is unavailable and navigating to the Replication tab. #5146
• Backups should take provider-level environment variables, including home directory. #5064
• Add hostname and display in the Nodes page along with node name and IP address. #4760
• Support the option of DNS names instead of IP addresses for nodes. Add option in Admin UI to choose between using hostnames or IP addresses for the on-premises provider. #4951 #4950
• Disable glob before running cleanup of old log files using zip_purge_yb_logs.sh. Fixes issue on Red Hat. #5169
• Fix Replication graph units and missing graph in Replication tab when metrics exist. #5423

Note:
Prior to version 2.0, YSQL was still in beta. As a result, the 2.0 release included a backward-incompatible file format change for YSQL. If you have an existing cluster running releases earlier than 2.0 with YSQL enabled, then you will not be able to upgrade to version 2.0+. Export from your existing cluster and then import into a new cluster (v2.0 or later) to use existing data.