Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money.

A background

Restic is an open source, free, fast, efficient, and secure cross-platform backup tool developed in GO. Restic uses encryption to ensure the security and integrity of your data, encrypting local data and transferring it to a specific storage.

Restic also supports incremental backups, which can be backed up and restored at any time. Restic supports most major operating systems, such as Linux, macOS, Windows, and smaller operating systems such as FreeBSD and OpenBSD.

Two restic profile

2.1 Restic Supported types

  • The local store
  • SFTP
  • REST Server
  • Amazon S3
  • Minio Server
  • OpenStack Swift
  • Backblaze B2
  • Microsoft Azure Blob Storage
  • Google Cloud Storage
  • Rclone mounted storage (e.g., Google Drive, OneDrive, etc.)

2.2 Restic and RcloneThe difference between

2.2.1 similarities

  • Both are command-line open source file synchronization and backup tools.
  • Both support backing up files to local, remote servers, or object stores.

2.2.2 similarities and differences between

  • Rclone aims at file synchronization, that is, ensuring the consistency of files at both ends and incremental backup.
  • Restic is for file backup and encryption. Files are encrypted and then transferred for backup, and it is incremental backup, that is, only the changed parts are backed up at a time.
  • The Rclone repository configuration is stored locally, and backup files are synchronized to the repository as they are.
  • Restic configuration information is written directly to the warehouse, which can be operated on any Restic installed computer as long as you have the warehouse password.
  • Rclone does not record file versions and cannot retrieve files at a specific point in time based on a single backup.
  • Restic generates a snapshot for each backup and records the file structure at the current point in time. You can retrieve the files at a specific point in time.
  • Rclone can transfer files between multiple configured storage devices.

In general, Rclone and Restic have different strengths and should be used according to different business needs. For example, Resitc is suitable for incremental backups of website data. Rclone is suitable for remote backup and archiving of regular files.

2.3 Restic design Principles

Restic is a correctly backed up program designed to follow the following principles:

  • Simplicity: Backup should be a smooth process, or you might want to skip it. Restic should be easy to configure and use so that in the event of data loss, you can directly recover it. Again, recovering data should not be complicated.
  • Fast: Backing up your data with Restic should only be limited by your network or hard drive bandwidth, so you can back up your files every day. If it takes too long, no one will back it up. Restore backups should transfer only the data needed for the files to be restored, so the process is also fast.
  • Verifiable: Recovery is more important than backup, so Restic makes it easy to verify that all data can be recovered.
  • Security: Restic uses encryption to keep your data confidential and complete. Assume that backup data is stored in a location that is not a trusted environment (for example, the shared space where someone else, such as a system administrator, has access to your backup). Restic is designed to protect your data from such attackers.
  • Efficient: As the data grows, additional snapshots should only occupy the storage of the actual increment. More importantly, duplicate data should be de-duplicated before it is actually written to the storage back end to save valuable backup space.

2.4 Related Terms

  • Repository: All data generated during backup is sent and stored in the Repository in a structured form, such as a file system hierarchy with multiple subdirectories. The repository implementation must be able to do many things, such as listing content. Storage services supported by V0.12.0 include: Aws S3, Minio Server, Wasabi, Aliyun OSS, OpenStack Swift, Backlbaze B2, Azure Blob Storage, Google Cloud Storage, RClone *

  • Blob: Blob combines multiple bytes of data with identifying information (such as the sha-256 hash of the data and its length), encrypted blocks of data, and metadata, where the metadata includes length, sha-256 hash information. Data blocks can store file data or directory structure data. Blobs range in size from 512 kibs to 8 mibs, so files smaller than 512KB are not split. Restic’s implementation goal is an average Blob size of 1MiB.

  • Pack: A package combines one or more BLOBs, for example, in a single file. A single data file in Restic, consisting of one or more BLOBs, is not modified once created.

    The prune operation deletes data that is no longer referenced.

  • Snapshot: Indicates the status of files or directories backed up at a point in time. State here refers to content and metadata, such as the name and modification time of a file or directory and its contents.

  • Storage ID: the SHA256 hash value of the Pack file by which the required data files can be loaded in the repository. Restic uses this ID as the file name for Pack, which is the SHA256 hash of the file. The design of the Pack file name, or hash value, also makes it easy to check if the data file has been changed.

Three installation restic

3.1 yum install

yum install yum-plugin-copr
yum copr enable copart/restic
yum install restic

Copy the code

3.2 docker installation

docker pull restic/restic
Copy the code

For more information: github.com/Lobaro/rest…

3.3 Source Code Installation

$ git clone https://github.com/restic/restic

$ cd restic

$ go run build.go
Copy the code

3.4 Configuring Automatic Completion

$ sudo ./restic generate --bash-completion /etc/bash_completion.d/restic
Copy the code

Four practical

Call the place where you keep your backups a repository. This chapter explains how to create (” init “) such a repository. Repositories can be stored locally or on remote servers or servers.

4.1 SFTP Inter-host Backup

4.1.1 Key free trust between hosts

To back up data from host A to host B, the data from host A to host B must be keyless and trust free

Ssh-keygen -t rsa ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]Copy the code

4.1.2 Creating A Backup File on Server A

Initial backup, /data is server B directory.

Viewing server B

4.1.3 Backup Operations

  • Performing data Backup
Restic -r SFTP :[email protected]:/data backup./Copy the code
  • Check the backup
Restic -r SFTP: [email protected]: / data snapshotsCopy the code
  • Viewing backup Content
Restic -r SFTP :[email protected]:/data ls 875a2a32Copy the code
  • Restore the snapshot
Restic-r SFTP :[email protected]:/data restore 875a2a32 -t./ restic-r SFTP :[email protected]:/data restore 875a2a32 --target ./Copy the code

  • Delete the backup
Restic-r SFTP :[email protected]:/data forget 875a2a32Copy the code

4.1.4 without password

The –password-file parameter is also used to automatically read the password.

Echo 'xxzx@789' > /root/resticpasswd # add --password-file to backup file Restic -r SFTP :[email protected]:/data --verbose backup./ --password-file /root/resticpasswdCopy the code

4.2 Object Storage Backup

Supports back-end object storage based on THE S3 protocol, such as Minio or Tencent/Ali object storage

4.2.1 AliCloud Object Storage

$ export AWS_ACCESS_KEY_ID=<YOUR-OSS-ACCESS-KEY-ID>
$ export AWS_SECRET_ACCESS_KEY=<YOUR-OSS-SECRET-ACCESS-KEY>

$ ./restic -o s3.bucket-lookup=dns -o s3.region=<OSS-REGION> -r s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME> init
$ restic -o s3.bucket-lookup=dns -o s3.region=oss-eu-west-1 -r s3:https://oss-eu-west-1.aliyuncs.com/bucketname init

restic -o s3.bucket-lookup=dns -o oss-cn-beijing.aliyuncs.com -r s3:https://xueltestoss.oss-cn-beijing.aliyuncs.com init

Copy the code
  • Create the repository
export AWS_ACCESS_KEY_ID=LTAIxxxxxxxdZa9
export AWS_SECRET_ACCESS_KEY=XvHxxxxxxxxxxxxxxxxxJt3wb7
restic -o s3.bucket-lookup=dns -o s3.region=oss-cn-beijing.aliyuncs.com -r s3:https://xueltestoss.oss-cn-beijing.aliyuncs.com/xueltestoss init
Copy the code

Object stores files

  • No key
#Save the password, such as Moerats, in the /root/resticpasswd text
echo 'xxzx@789' > /root/resticpasswd
#Then add the --password-file parameter to the backup command to read the password in the text. In this example, SFTP is used
Copy the code
  • Perform a backup
restic -r s3:https://oss-cn-beijing.aliyuncs.com/xueltestoss --password-file /root/resticpasswd backup /data/
Copy the code

Other recovery operations are basically the same as those for SFTP.

other

Restic is a great data backup solution, RClone is a great data synchronization solution, and Minio is a great data store integration.

Refer to the article

  • Restic. Readthedocs. IO/en/v0.12.0 /…
  • Github.com/restic/rest…
  • restic.net/