Tool for easy ClickHouse backup and restore using object storage for backup files.
OTHER License
Bot releases are hidden (Show)
Published by AlexAkulov over 3 years ago
IMPROVEMENTS
BUG FIXES
Published by AlexAkulov over 3 years ago
It is the last release of v0.x.x
IMPROVEMENTS
BUG FIXES
Published by AlexAkulov over 3 years ago
IMPROVEMENTS
compression_format: none
optionBROKEN CHANGES
Published by AlexAkulov over 3 years ago
IMPROVEMENTS
CLICKHOUSE_AUTO_CLEAN_SHADOW
option for cleaning shadow folder before backup. Enabled by default.CLICKHOUSE_SYNC_REPLICATED_TABLES
option for sync replicated tables before backup. Enabled by default.BUG FIXES
Published by AlexAkulov almost 4 years ago
IMPROVEMENTS
BUG FIXES
Published by AlexAkulov almost 4 years ago
IMPROVEMENTS
Added integration with clickhouse cli. Now backups can be managed via clickhouse-client
How it works:
clickhouse-backup server
backup_actions
and backup_list
tablesCREATE TABLE system.backup_actions (command String, start DateTime, finish DateTime, status String, error String) ENGINE=URL('http://127.0.0.1:7171/backup/actions', JSONEachRow)
CREATE TABLE system.backup_list (name String, created DateTime, size Int64, location String) ENGINE=URL('http://127.0.0.1:7171/backup/list', JSONEachRow)
INSERT INTO system.backup_actions (command) VALUES ('create backup1')
INSERT INTO system.backup_actions (command) VALUES ('upload backup1')
SELECT * FROM system.backup_list
┌─name───────────┬─────────────created─┬──size─┬─location─┐
│ backup1 │ 2020-12-01 08:43:47 │ 0 │ local │
│ backup1.tar.gz │ 2020-12-01 08:43:54 │ 74249 │ remote │
└────────────────┴─────────────────────┴───────┴──────────┘
SELECT * FROM system.backup_actions
┌─command────────┬───────────────start─┬──────────────finish─┬─status──┬─error─┐
│ create backup1 │ 2020-12-01 09:01:02 │ 2020-12-01 09:01:02 │ success │ │
│ upload backup1 │ 2020-12-01 09:01:05 │ 2020-12-01 09:01:05 │ success │ │
└────────────────┴─────────────────────┴─────────────────────┴─────────┴───────┘
Published by AlexAkulov almost 4 years ago
IMPROVEMENTS
BUG FIXES
Published by AlexAkulov about 4 years ago
IMPROVEMENTS
Published by AlexAkulov over 4 years ago
IMPROVEMENTS
CLICKHOUSE_TIMEOUT
and CLICKHOUSE_FREEZE_BY_PART
options. It's useful when your tables have too many parts and default freezing doesn't work.BUG FIXES
Published by AlexAkulov almost 5 years ago
BUG FIXES
IMPROVEMENTS
S3_DISABLE_CERT_VERIFICATION
option (thanks @martenlindblad)Published by AlexAkulov almost 5 years ago
IMPROVEMENTS
BROKEN CHANGES
Published by AlexAkulov about 5 years ago
IMPROVEMENTS
Published by AlexAkulov about 5 years ago
IMPROVEMENTS
Comparison table of upload 31GiB data with v0.4.0:
+------+----------------+----------------+
| | v0.4.0 | v0.4.1 |
+------+----------------+----------------+
| tar | Time: 8m14s | Time: 7m50s |
| | Size: 30.8GB | Size: 30.8GB |
+------+----------------+----------------+
| lz4 | Time: 22m50s | Time: 15m15s |
| | Size: 24GB | Size: 24.5GB |
+------+----------------+----------------+
| gzip | Time: 13m21s | Time: 6m54s |
| | Size: 21GB | Size: 24.2GB |
+------+----------------+----------------+
Published by AlexAkulov about 5 years ago
IMPROVEMENTS
DEPREСATIONS
BUG FIXES
Published by AlexAkulov over 5 years ago
IMPROVEMENTS
Published by AlexAkulov over 5 years ago
IMPROVEMENTS
Published by AlexAkulov over 5 years ago
IMPROVEMENTS
clickhouse-backup create
clickhouse-backup upload --diff-from=$(clickhouse-backup list local penult) $(clickhouse-backup list local latest)
clickhouse-backup delete local <backup_name>
clickhouse-backup delete s3 $(clickhouse-backup list s3 penult)
Published by AlexAkulov over 5 years ago
Added AWS IAM support
Published by AlexAkulov over 5 years ago
BUG FIXES
Published by AlexAkulov over 5 years ago
BUG FIXES
DEPREСATIONS
IMPROVEMENTS
+-----+---------------------+---------------------+
| | v0.3.1 | v0.3.2 |
+-----+---------------------+---------------------+
| tar | Time: 45m:58s | Time: 14m:34s |
| | Disk Usage: 70% | Disk Usage: 58% |
| | Network Util: 100Mb | Network Util: 300Mb |
+-----+---------------------+---------------------+
| lz4 | Time: 28m:47s | Time: 26m:35s |
| | Disk Usage: 30% | Disk Usage: 26% |
| | Network Util: 120Mb | Network Util: 130Mb |
+-----+---------------------+---------------------+