Contents of Articles:
- Introduction
- Installing Restic And Connecting To Storage
- Example Of Working With Storage via Rustic
- Conclusion
Introduction
Many administrators set up backups while using the Rsync Utility in 2007. It was necessary to allocate a separate server for storage. And one of the frequent problems was the costs increasing due to reserving a server specifically for backups. Devices that were backed up were located next to the backup host. This violated the 3-2-1 rule, which recommends having backups in at least three locations, two of which are on different media and one offsite. You need to have hosts in different data centers to build a truly reliable system.
Today, you can contact 3HCloud to use cloud data storage services. Below we will tell you how to use the Restic utility. That is a backup system that provides tools for storing backups in a versioned repository. Restic also allows you to define rules for including and excluding files and directories in the backup.
Installing Restic And Connecting To Storage
We will use the Ubuntu 22.04 distribution with Restic installation from the repository in this example.
-
Connect to your server and install the necessary dependencies.
$ sudo apt update
$ sudo apt install -y restic -
Update Restic and check if the required version is installed.
restic self-update
$ restic version
restic 0.15.2 compiled with go1.20.3 on linux/amd64 -
Export the environment variables and initialize the repository using 3HCloud S3 as a backend.
$ export AWS_ACCESS_KEY_ID=<container_username>
$ export AWS_SECRET_ACCESS_KEY=<container_password>
$ restic -r s3:s3.dfw.3hcloud.com/<container_name> init
Set the password for the repository after entering the command and displaying the message in the terminal.
Example Of Working With Storage via Rustic
It’s necessary to know how to work with a storage object while using the utility.
Backup
Let's create a directory with a file that we will copy. If you already have a directory with data for a future backup, you can skip this step.
$ mkdir dir
$ echo line1 > dir/file1.txt
$ cat dir/file1.txt
line1
Let's start creating a backup in dry-run mode to make sure that everything will work correctly. You will need to enter the password from the repository that you set earlier during the execution of the command.
restic -r s3:s3.dfw.3hcloud.com/<container_name> backup -vv --dry-run <directory>
The terminal will output a message similar to the following:
repository e4218f76 opened (version 1)
created new cache in /root/.cache/restic
lock repository
no parent snapshot found, will read all files
load index files
start scan on [/root/dir]
start backup on [/root/dir]
scan finished in 0.243s: 1 files, 6 B
new /root/dir/file1.txt, saved in 0.007s (6 B added)
new /root/dir/, saved in 0.008s (0 B added, 0 B stored, 425 B metadata)
new /root/, saved in 0.008s (0 B added, 0 B stored, 419 B metadata)
Files: 1 new, 0 changed, 0 unmodified
Dirs: 2 new, 0 changed, 0 unmodified
Data Blobs: 1 new
Tree Blobs: 3 new
Would add to the repository: 1.034 KiB (1.304 KiB stored)
processed 1 files, 6 B in 0:00
Now you can remove the –dry-run key and make the first backup.
restic -r s3:s3.dfw.3hcloud.com/<container_name> backup <directory>
repository e4218f76 opened (version 1)
no parent snapshot found, will read all files
Files: 1 new, 0 changed, 0 unmodified
Dirs: 2 new, 0 changed, 0 unmodified
Added to the repository: 1.034 KiB (1.304 KiB stored)
processed 1 files, 6 B in 0:00
snapshot a7d06fa4 saved
You can view the list of backups (snapshots) while using the special snapshots command. In order not to enter a password every time, you can write it to a file and use the -p key.
$ echo 'pass' > backup_password
$ restic -r s3:s3.dfw.3hcloud.com/<container_name> -p backup_password snapshots
enter password for repository:
repository e4218f76 opened (version 1)
ID Time Host Tags Paths
----------------------------------------------------------------
a7d06fa4 2023-06-14 16:49:09 www /root/dir
----------------------------------------------------------------
1 snapshots
Now let's make changes to the directory, repeat the backup process, and see how it looks.
$ echo line2 >> dir/file1.txt
$ echo line1 >> dir/file2.txt
$ ls -l dir/
total 8
-rw-r--r-- 1 root root 12 Jun 14 16:52 file1.txt
-rw-r--r-- 1 root root 6 Jun 14 16:52 file2.txt
$ restic -r s3:s3.dfw.3hcloud.com/<container_name> -p backup_password backup ~/dir
repository e4218f76 opened (version 1)
using parent snapshot a7d06fa4
Files: 1 new, 1 changed, 0 unmodified
Dirs: 0 new, 2 changed, 0 unmodified
Added to the repository: 1.377 KiB (1.646 KiB stored)
processed 2 files, 18 B in 0:00
snapshot e0f0801e saved
$ restic -r s3:s3.dfw.3hcloud.com/restic_backup -p backup_password snapshots
repository e4218f76 opened (version 1)
ID Time Host Tags Paths
----------------------------------------------------------------
a7d06fa4 2023-06-14 16:49:09 www /root/dir
e0f0801e 2023-06-14 16:53:02 www /root/dir
----------------------------------------------------------------
2 snapshots
After backing up the modified directory, a new copy appeared in the object storage.
Restoring A Backup
Restoring backups using Restic looks like this:
restic -r s3:s3.dfw.3hcloud.com/<container_name> restore <id_snapshot> --target <directory>
Old Copy Deletion Policy
To delete old backups, you can use the forget command. You can read more about it in the
manual to configure the system in compliance with your demands.
Let's consider an example where we need to store daily backups for the last week, weekly backups for the last month, monthly backups for the last year, and yearly backups for the last 75 years. In this case, arguments of the following format can be passed:
forget --keep-within-daily 7d --keep-within-weekly 1m --keep-within-monthly 1y --keep-within-yearly 75y
Let's consider an example of using the forget command:
$ restic -r s3:s3.dfw.3hcloud.com/restic_backup -p backup_password forget --keep-within-daily 7d --keep-within-weekly 1m --keep-within-monthly 1y --keep-within-yearly 75y
repository e4218f76 opened (version 1)
Applying Policy: keep daily snapshots within 7d, weekly snapshots within 1m, monthly snapshots within 1y, yearly snapshots within 75y
keep 1 snapshots:
ID Time Host Tags Reasons Paths
-------------------------------------------------------------------------------
e0f0801e 2023-06-14 16:53:02 www daily within 7d /root/dir
weekly within 1m
monthly within 1y
yearly within 75y
-------------------------------------------------------------------------------
1 snapshots
remove 1 snapshots:
ID Time Host Tags Paths
------------------------------------------------------------
a7d06fa4 2023-06-14 16:49:09 www /root/dir
------------------------------------------------------------
1 snapshots
[0:00] 100.00% 1 / 1 files deleted
Automation
Combining commands for backup creation and deletion (rotation) allows for process automation. For instance, by placing the following task in cron:
export AWS_ACCESS_KEY_ID=<container_username>
export AWS_SECRET_ACCESS_KEY=<container_password>
restic -r s3:s3.dfw.3hcloud.com/<container_name> -p backup_password backup <directory> 2>&1 | systemd-cat -t restic-cron-backup
Logging via systemctl-cat enables easy access to backup logs when needed:
$ journalctl -t restic-cron-backup
Nov 16 23:05:57 www restic-cron-backup[9390]: using parent snapshot 25f3eb23
Nov 16 23:05:57 www restic-cron-backup[9390]: Files: 0 new, 0 changed, 2 unmodified
Nov 16 23:05:57 www restic-cron-backup[9390]: Dirs: 0 new, 0 changed, 2 unmodified
Nov 16 23:05:57 www restic-cron-backup[9390]: Added to the repository: 0 B (0 B stored)
Nov 16 23:05:57 www restic-cron-backup[9390]: processed 2 files, 18 B in 0:00
Nov 16 23:05:57 www restic-cron-backup[9390]: snapshot 5bfd0c72 saved
Nov 16 23:06:10 www restic-cron-backup[9458]: Applying Policy: keep daily snapshots within 7d, weekly snapshots within 1m, monthly snapshots within 1>
Nov 16 23:06:10 www restic-cron-backup[9458]: keep 1 snapshots:
Nov 16 23:06:10 www restic-cron-backup[9458]: ID Time Host Tags Reasons Paths
Nov 16 23:06:10 www restic-cron-backup[9458]: -----------------------------------------------------------------------------------
Nov 16 23:06:10 www restic-cron-backup[9458]: 5bfd0c72 2023-11-16 23:05:56 www daily within 7d /root/dir
Nov 16 23:06:10 www restic-cron-backup[9458]: weekly within 1m
Nov 16 23:06:10 www restic-cron-backup[9458]: monthly within 1y
Nov 16 23:06:10 www restic-cron-backup[9458]: yearly within 75y
Nov 16 23:06:10 www restic-cron-backup[9458]: -----------------------------------------------------------------------------------
Nov 16 23:06:10 www restic-cron-backup[9458]: 1 snapshots
Nov 16 23:06:10 www restic-cron-backup[9458]: remove 2 snapshots:
Nov 16 23:06:10 www restic-cron-backup[9458]: ID Time Host Tags Paths
Nov 16 23:06:10 www restic-cron-backup[9458]: ----------------------------------------------------------------
Nov 16 23:06:10 www restic-cron-backup[9458]: e0f0801e 2023-11-16 22:53:02 www /root/dir
Nov 16 23:06:10 www restic-cron-backup[9458]: 25f3eb23 2023-11-16 23:05:51 www /root/dir
Nov 16 23:06:10 www restic-cron-backup[9458]: ----------------------------------------------------------------
Nov 16 23:06:10 www restic-cron-backup[9458]: 2 snapshots
Nov 16 23:06:10 www restic-cron-backup[9458]: [0:00] 100.00% 2 / 2 files deleted
Conclusion
We have a convenient and reliable tool for working with backups and providing the necessary storage depth with the help of Restic and 3HCloud object storage. This approach helps reduce the costs associated with setting up and maintaining your own system, providing access to archives from anywhere in the world.
