backup

Application that makes an encrypted backup of photo/music files.

The output is encrypted with GPG and includes a restore script that can verify backup integrity.

The script is intended for personal use. You are welcome to use it, but do not expect support in any form.

Actions Build Pipeline

The hosted Actions build pipeline uses an Ubuntu 18.04.

To reproduce the Actions environment locally:

$ ./gradlew makeActionsDockerImage
$ podman run -i -t actions-backup

Jotta Verification

With Jotta it is possible to get MD5 sums of stored files like this:

$ jotta-cli ls -l Archive/backup/music
Name                           Size                          Checksum                LastModified  
-------------------------  --------  --------------------------------  --------------------------  
music-2021.03.07-01.crypt   1.00GiB  7fcf5071496b4d2a6aa981caf9adbec8  2021-03-07-T12:37:20Z+0100  
...

The verify -j path option allows the backup info in the cloud to be verified by comparing the file checksums.

Build local jar

$ ./gradlew shadowJar -Pversion=1.2.1
$ cp build/libs/backup-shadow.jar ~/bin/backup-1.2.2.jar

Testing

Test Certificate

The tests use a dummy certificate.

A new one can be generated by running the script ./src/test/create-test-key.sh.

Design

Data format

The backup set data format has three levels:

crypt : crypt-container : root-elements

crypt is the encrypted output file(s). This is the representation on disk of the backup set.

Inside the crypt is a single crypt-container with all the root-elements of the source folder. This is a simple tar archive.

Each of the root-elements are single-files; directories are packed into a tar-archive, regular files appear raw.

The restore script can determine the source file type by it's prefix; if the file name starts with "./" it is a archived folder (the name will also have the suffix .tar).

Output Types

Given this input file tree:

.
├── dir-a
│   ├── file-a1.bin
│   └── file-a2.bin
├── dir-b
│   └── file-b1.bin
├── file-root1.bin
└── file-root2.txt

Size-limited, numbered encrypted files

A single encrypted stream is created which is split over a number of output files (determined by size option).

In the encrypted stream a single tar-archive holds the backup root elements.

Directories are packed into tar-files before they are added.

target1.crypt + target2.crypt...            (crypt(s))
└── target.tar                              (crypt-container)
    ├── dir-a.tar       [^untar]            (root element dir)
    │   └── dir-a
    │      ├── file-a1.bin
    │      └── file-a2.bin
    ├── dir-b.tar       [^untar]            (root element dir)
    │   └── dir-b
    │      └── file-b1.bin
    ├── file-root1.bin  [^cat]              (root element file)
    └── file-root2.txt  [^cat]              (root element file)

Directory tar-files are untar'd on restore, regular files cat'd in place on restore.