v2.5.1

BUG FIXES

• fixed issue when set both AWS_ROLE_ARN and S3_ASSUME_ROLE_ARN then S3_ASSUME_ROLE_ARN shall have more priority than AWS_ROLE_ARN fix 898

v2.5.0

IMPROVEMENTS

• complete removed support for legacy backups, created with version prior v1.0
• removed disable_progress_bar config option and related progress bar code
• added --delete-source parameter for upload and create_remote commands to explicitly delete local backup during upload, fix 777
• added support for --env ENV_NAME=value cli parameter for allow dynamically override any config parameter, fix 821
• added support for use_embedded_backup_restore: true with empty embedded_backup_disk value, tested on S3/GCS over S3/AzureBlobStorage, fix 695
• --rbac, --rbac-only, --configs, --configs-only now works with use_embedded_backup_restore: true
  -- --data for restore with use_embedded_backup_restore: true will use allow_non_empty_tables=true to allow fix 756
• added --diff-from-remote parameter for create command, will copy only new data parts object disk data, also allows to download properly object disk data from required backup during restore, fix 865
• added support of native Clickhouse incremental backup for use_embedded_backup_restore: true fix 735
• added GCS_CHUNK_SIZE config parameter, try to speedup GCS upload fix 874, thanks @dermasmid
• added --remote-backup cli parameter to tables command and GET /backup/table, fix 778
• added rbac_always_backup: true option to default config, will create backup for RBAC objects automatically, restore still require --rbac to avoid destructive actions, fix 793
• added rbac_conflict_resolution: recreate option for RBAC object name conflicts during restore, fix 851
• added upload_max_bytes_per_seconds and download_max_bytes_per_seconds config options to allow throttling without CAP_SYS_NICE, fix 817
• added clickhouse_backup_in_progress_commands metric, fix 836
• switched to golang 1.22
• updated all third-party SDK to latest versions
• added clickhouse/clickhouse-server:24.3 to CI/CD

BUG FIXES

• continue S3_MAX_PARTS_COUNT default value from 2000 to 4000 to continue decrease memory usage for S3
• changed minimal part size for multipart upload in CopyObject from 5Mb to 10Mb
• restore SQL UDF functions after restore tables
• execute ALTER TABLE ... DROP PARTITION instead of DROP TABLE for restore and restore_remote with parameters --data --partitions=..., fix 756
• fix wrong behavior for freeze_by_part + freeze_by_part_where, fix 855
• apply CLICKHOUSE_SKIP_TABLES_ENGINES during create command
• fixed behavior for upload / download when .inner. table missing for MATERIALIZED VIEW by table pattern, fix 765
• fixed ObjectDisks + CLICKHOUSE_USE_EMBEDDED_BACKUP_RESTORE: true - shall skip upload object disk content, fix 799
• fixed connection to clickhouse-server behavior when long clickhouse-server startup time and docker-entrypoint.d processing, will infinite reconnect each 5 seconds, until success, fix 857
• fixed USE_EMBEDDED_BACKUP_RESTORE=true behavior to allow use backup disk with type local, fix 882
• fixed wrong list command behavior, it shall scann all system.disks path not only default disk to find pratially created backups, fix 873
• fixed create --rbac behavior, don't create access folder if no RBAC objects is present
• fixed behavior when system.disks contains disk which not present in any storage_policies, fix 845

v2.4.35

IMPROVEMENTS

• set part size for s3:CopyObject minimum 128Mb, look details https://repost.aws/questions/QUtW2_XaALTK63wv9XLSywiQ/s3-sync-command-is-slow-to-start-on-some-data

v2.4.34

BUG FIXES

• fixed wrong behavior for CLICKHOUSE_SKIP_TABLES_ENGINES for engine=EngineName without parameters

v2.4.33

BUG FIXES

• fixed wrong anonymous authorization for service account for GCS, added GCS_SKIP_CREDENTIALS fix 848, fix 847, thanks @sanadhis

v2.4.32

IMPROVEMENTS

• added ability to make custom endpoint for GCS, fix 837, thanks @sanadhis

BUG FIXES

• fixed wrong config validation for object_disk_path even when no object disk present in backup during restore, fix 842

v2.4.31

IMPROVEMENTS

• add check_sum_algorithm parameter for s3 config section with "" default value, to avoid useless CPU usage during upload to S3 storage, additional fix 829
• upload will delete local backup if upload successful, fix 834

BUG FIXES

• fix miss checksum for CopyObject in s3, fix 835, affected 2.4.30
• fix wrong behavior for restore --rbac-only and restore --rbac for backups which not contains any schema, fix 832

v2.4.30

BUG FIXES

• fix download command corner cases for increment backup for tables with projections, fix 830
• more informative error during try to restore not exists local backup
• fix upload command for S3 when object lock policy turned on, fix 829

v2.4.29

IMPROVEMENTS

• add AZBLOB_DEBUG environment and debug config parameter in azblob section

BUG FIXES

• force set RefCount to 0 during restore for parts in S3/GCS over S3/Azure disks, for properly works DROP TABLE / DROP DATABASE
• use os.Link instead os.Rename for ClickHouse 21.4+, to properly create backup object disks
• ignore frozen_metadata during, create, upload, download and restore commands, fix 826
• allow_parallel: true doesn't work after execute list command, fix 827
• fix corner cases, when disk have encrypted type and underlying disk is object storage

v2.4.28

IMPROVEMENT

• refactoring watch command, after https://github.com/Altinity/clickhouse-backup/pull/804

BUG FIXES

• fix deletion for object_disk_path and embedded backups, after upload to properly respect backups_to_keep_remote

v2.4.27

BUG FIXES

• fix deletion for object_disk_path (all backups with S3, GCS over S3, AZBLOB disks from 2.4.0-2.4.25 didn't delete properly their data from backup bucket)

v2.4.26

IMPROVEMENTS

• improve re-balance disk during download if disk not exists in system.disks. Use least used for local disks and random for object disks, fix 561
  BUG FIXES
• fix regression check_parts_columns for Enum types (2.4.24+), fix 823
• properly applying marcos to object_disk_path during delete
• fix deletion for object_disk_path (all backups with S3, GCS over S3, AZBLOB disks from 2.4.0-2.4.25 didn't delete properly their data from backup bucket)

v2.4.25

BUG FIXES

• fix --restore-table-mapping corner cases for when destination database contains special characters , fix 820

v2.4.24

BUG FIXES

• fix check_parts_columns corner cases for AggregateFunction and SimpleAggregateFunction versioning, fix 819

v2.4.23

IMPROVEMENTS

• refactoring of restore command to allow parallel execution of ALTER TABLE ... ATTACH PART and improve parallelization of CopyObject during restore.

v2.4.22

BUG FIXES

• change S3_MAX_PARTS_COUNT default value from 256 to 2000 to fix memory usage for s3 which increased for 2.4.16+

v2.4.21

BUG FIXES

• refactoring execution UpdateBackupMetrics, to avoid context canceled error, fix 814

v2.4.20

IMPROVEMENTS

• refactoring of create command to allow parallel execution of FREEZE and UNFREEZE and table level parallelization object_disk.CopyObject
• add CLICKHOUSE_MAX_CONNECTIONS config parameter to allow parallel execution FREEZE / UNFREEZE
• change go.mod to allow GO111MODULE=on go install github.com/Altinity/clickhouse-backup/v2/cmd/clickhouse-backup@latest

v2.4.19

BUG FIXES

• use single s3:CopyObject call instead s3:CreateMultipartUpload+s3:UploadCopyPart+s3:CompleteMultipartUpload for files with size less 5Gb

v2.4.18

BUG FIXES

• remove HeadObject request to calculate source key size in CopyObject, to allow cross region S3 disks backup, fix 813
• make optional /backup/kill query parameter command and optional arguments for kill command handled via /backup/actions, if omitted then will kill first command in "In progress" status, fix 808