|
|
|
|
# zfs-snapshot-backup
|
|
|
|
|
|
|
|
|
|
**Automatically create (incremental) backups of zfs snapshots on a file server.**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This is currently tailored only to our setup of a FreeNAS in combination with
|
|
|
|
|
our virtual server and storage provided by the Rechenzentrum.
|
|
|
|
|
|
|
|
|
|
The script will create backups of ZFS snapshots for elab journals. The name of
|
|
|
|
|
the ZFS dataset (from wich the snapshots are created) must start with `elabfs-`
|
|
|
|
|
followed by the elab member name, e.g. `LukasMetzler`
|
|
|
|
|
|
|
|
|
|
The snapshots should be made with the `periodic snapshot task` provided by the
|
|
|
|
|
FreeNAS system. Snapshots should be kept on the NAS for a couple of days
|
|
|
|
|
(currently one week) to allow transfer errors not to interfere with the backup
|
|
|
|
|
process.
|
|
|
|
|
|
|
|
|
|
The backup process should be run after a snapshot is taken. Proposed is to take
|
|
|
|
|
a daily snapshot at night (currently between 2am and 3am) and start the backup
|
|
|
|
|
process later via cron (currently set to 4am).
|
|
|
|
|
|
|
|
|
|
The documentation in our wiki can be found at:
|
|
|
|
|
https://wiki.cpi.imtek.uni-freiburg.de/CPIvServerDocumentation/ElabOnZfs
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## restoring backups ##
|
|
|
|
|
|
|
|
|
|
1. copy a snapshot backup to the NAS
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
cd /mnt/Datenspeicher/snap-backup-dataset/temporary-backups/
|
|
|
|
|
scp -i ../backup_key zfs_snap_backup@etha.cpi.imtek.uni-freiburg.de:~/zfs-backups/<backup-file> tmp-backup-file.gz
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
2. unzip the backup file and restore it into a dataset
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
gunzip -c tmep-backup-file.gz | zfs receive -F Datenspeicher/test-backup
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
3. remove the (local) backup file
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
rm tmp-backup-file.gz
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## notes on implementation ##
|
|
|
|
|
|
|
|
|
|
At first I tried to implement the complete process in one python file, but this
|
|
|
|
|
lead to an unforseen (and very strange) error:
|
|
|
|
|
|
|
|
|
|
The created gzip file was not completely written before copying it to the remote
|
|
|
|
|
server. Even closing the file descriptor manually or waiting for a very long
|
|
|
|
|
time would not solve the problem. The gzip was only completed if the python
|
|
|
|
|
process was stopped.
|
|
|
|
|
|
|
|
|
|
Therefore creating the backups and copying them to the remote server is now a
|
|
|
|
|
two step process, combined in the `run_snapshot_backups.sh` shell script.
|
|
|
|
|
|
|
|
|
|
I have the idea, that it has todo with the copy on write feature of zfs, but
|
|
|
|
|
I'm not sure about it.
|
