The off-site copy, and proving your backups still restore

GOING FURTHER

Tested for real: I ran every command in this guide top-to-bottom on a fresh Ubuntu 24.04 virtual machine (4 CPUs, 8 GB RAM) on 13 August 2026. The run continued directly from the end state of Backups while you sleep. Versions at test time: Ubuntu 24.04.4 LTS, restic 0.16.4, systemd 255.

Found a problem? Tell me via the contact page.

An honest note before anything else. The off-site leg was tested against a localhost stand-in: this same machine talking to its own SSH server, with localhost standing in for a real second box. Every command and every output below is real. For a genuine remote, exactly two things change: the hostname in the address (localhost becomes your remote's name or IP), and how your key gets there (ssh-copy-id YOUR-SERVER-USER@YOUR-SERVER-IP, exactly as in SSH keys, properly).

Placeholders: anything written in CAPS-WITH-DASHES, like YOUR-SERVER-IP, is a value you replace with your own. $USER is different — leave it exactly as written; the shell fills in your username for you, as explained below.

What you'll end up with

Your nightly backup pushes a copy to a second machine. And a one-word command proves the backup still restores, because an untested backup rots quietly.

The finished state from my test run:

$ restore-drill
Summary: Restored 6 files/dirs (2.000 MiB) in 0:00
RESTORE DRILL: PASS — restored files match the originals

Who this is for

GOING FURTHER. You have done Backups while you sleep: systemd timers and retention and kept its end state. That means the nightly restic-backup service and timer, plus everything from the restic guide underneath. This guide starts exactly there.

Time and cost

The commands for this guide and the timers guide took me about 5 minutes in total on the test machine, measured as one run. Budget 40 minutes for this half. Cost: nothing with the localhost stand-in. When you want a real off-site machine, cheap VPS options exist — and it is entirely optional today.

Words you'll meet

• Off-site — a copy stored away from the machine — the copy that survives fire, theft, or a dead power supply. (basics)
• SFTP — file transfer over SSH. If you can SSH to a machine, restic can store backups on it. (basics)
• Journal — systemd's central log. journalctl reads it; everything our service prints lands there. (basics)
• Restore drill — a rehearsal restore into a scratch folder, compared against the originals, so you know the backup works before you need it. (basics)

Before you start

• The end state of the timers guide — service, timer, repository, password file, ~/my-files.
• Everywhere below, $USER fills in your username automatically as files are written — nothing to edit mid-line. Verify with the cat checks.

The steps

Step 1 — set up the off-site leg

A backup drive in the same room dies in the same fire. The fix is a copy on a different machine. Restic's SFTP backend means any machine you can SSH into will do.

Remember the stand-in note at the top: below, localhost plays the part of your remote. On a real remote, swap the hostname and use ssh-copy-id where marked.

Restic needs to SSH without a password prompt, so make a key with no passphrase:

ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519

Your output will vary, but look for:

Generating public/private ed25519 key pair.
Your identification has been saved in /home/tester/.ssh/id_ed25519
[...snipped...]

(If a key already exists it will ask before overwriting — answer n and use your existing key.)

For the localhost stand-in, trust our own key. On a real remote, replace these two commands with ssh-copy-id instead:

cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys

chmod 600 ~/.ssh/authorized_keys

Prove the passwordless login and accept the host's fingerprint in one go:

ssh -o StrictHostKeyChecking=accept-new $USER@localhost "echo ssh works"

You should see something like:

Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
ssh works

Make the folder the remote repository will live in:

mkdir -p ~/offsite

Now create the off-site repository. The --from-repo ... --copy-chunker-params part copies the local repository's chunking settings. That keeps deduplication working across the two repositories, so nightly copies stay small:

restic -r sftp:$USER@localhost:/home/$USER/offsite/restic-repo --password-file ~/.config/restic/password init --from-repo /mnt/backup-drive/restic-repo --from-password-file ~/.config/restic/password --copy-chunker-params

The important line is:

created restic repository 45db918e1e at sftp:tester@localhost:/home/tester/offsite/restic-repo with chunker parameters copied from secondary repository

Copy every local snapshot across:

restic -r sftp:$USER@localhost:/home/$USER/offsite/restic-repo --password-file ~/.config/restic/password copy --from-repo /mnt/backup-drive/restic-repo --from-password-file ~/.config/restic/password

Your output will vary, but look for:

snapshot ec6479d9 of [/home/tester/my-files] at 2026-08-13 16:35:59 [...snipped...]
  copy started, this may take a while...
[0:00] 100.00%  3 / 3 packs copied

snapshot 5891239c saved

snapshot 95cb8fa0 of [/home/tester/my-files] at 2026-08-13 16:35:12 [...snipped...]
  copy started, this may take a while...
[0:00]          0 packs copied

snapshot 0a3ac72f saved

Notice the second snapshot needed 0 packs copied — its data already arrived with the first. That is --copy-chunker-params paying off. Copied snapshots get new IDs on the far side; the dates are what match.

Check it worked. List the off-site repository's snapshots:

restic -r sftp:$USER@localhost:/home/$USER/offsite/restic-repo --password-file ~/.config/restic/password snapshots

You should see something like:

ID        Time                 Host            Tags        Paths
--------------------------------------------------------------------------------
0a3ac72f  2026-08-13 16:35:12  ubuntu-testbed              /home/tester/my-files
5891239c  2026-08-13 16:35:59  ubuntu-testbed              /home/tester/my-files
--------------------------------------------------------------------------------
2 snapshots

Two snapshots, timestamps matching your local repository. Your data now exists in two places.

Step 2 — put the off-site copy into the nightly run

Add the copy as a third step of the nightly service. This appends one ExecStart line to the service file from the timers guide:

sudo tee -a /etc/systemd/system/restic-backup.service > /dev/null <<EOF
ExecStart=/usr/bin/restic -r sftp:$USER@localhost:/home/$USER/offsite/restic-repo --password-file /home/$USER/.config/restic/password copy --from-repo /mnt/backup-drive/restic-repo --from-password-file /home/$USER/.config/restic/password
EOF

Reload and fire the whole pipeline once, by hand:

sudo systemctl daemon-reload

sudo systemctl start restic-backup.service

Check it worked. Read the last few journal lines:

sudo journalctl -u restic-backup.service --since "-2 min"

A passing run includes:

[...snipped: backup and retention steps...]
Aug 13 16:36:41 ubuntu-testbed restic[3356]:   copy started, this may take a while...
Aug 13 16:36:41 ubuntu-testbed restic[3356]: [0:00] 100.00%  1 / 1 packs copied
Aug 13 16:36:41 ubuntu-testbed restic[3356]: snapshot 56116612 saved
Aug 13 16:36:41 ubuntu-testbed systemd[1]: Finished restic-backup.service [...snipped...]

One start, three jobs: backup, retention, off-site copy. That is what runs at 02:30 every night from now on.

Step 3 — script the restore drill

Automation cuts both ways: nobody watches a nightly backup, so nobody notices it quietly going bad. The answer is a drill you can run in one word. It restores the newest snapshot to a scratch folder, compares every file against the originals, and says PASS or FAIL.

Write the script:

sudo tee /usr/local/bin/restore-drill > /dev/null <<"EOF"
#!/usr/bin/env bash
# Restore drill: prove the latest backup actually restores.
set -euo pipefail

REPO=/mnt/backup-drive/restic-repo
PWFILE=$HOME/.config/restic/password
SOURCE=$HOME/my-files
TARGET=/tmp/restore-drill

rm -rf "$TARGET"
restic -r "$REPO" --password-file "$PWFILE" restore latest --target "$TARGET"

if diff -r "$SOURCE" "$TARGET$SOURCE"; then
  echo "RESTORE DRILL: PASS — restored files match the originals"
else
  echo "RESTORE DRILL: FAIL — differences found; do not trust this backup until you know why"
  exit 1
fi
EOF

(The "EOF" quotes matter here — they stop the shell expanding $HOME while writing, because this script should expand it when it runs.)

Make it executable:

sudo chmod +x /usr/local/bin/restore-drill

Run your first drill:

restore-drill

You should see something like:

restoring <Snapshot bd2201e5 of [/home/tester/my-files] at 2026-08-13 16:36:37 [...snipped...]> to /tmp/restore-drill
Summary: Restored 6 files/dirs (2.000 MiB) in 0:00
RESTORE DRILL: PASS — restored files match the originals

Check it worked. The last line is RESTORE DRILL: PASS. Run this monthly. Put it in your calendar — the machine backs itself up; your job is to test restores.

Step 4 — the deep check: read every byte

The plain check from the restic guide audits the repository's structure. check --read-data goes further: it reads every byte of stored data back and verifies it. It is the closest thing to certainty that your disk has not silently corrupted anything.

Check the local repository:

time restic -r /mnt/backup-drive/restic-repo --password-file ~/.config/restic/password check --read-data

A passing run includes:

load indexes
check all packs
check snapshots, trees and blobs
[0:00] 100.00%  2 / 2 snapshots

read all data
[0:00] 100.00%  4 / 4 packs

no errors were found

real	0m0.788s

And the off-site one:

time restic -r sftp:$USER@localhost:/home/$USER/offsite/restic-repo --password-file ~/.config/restic/password check --read-data

The important line is:

[...snipped: same check stages...]
no errors were found

real	0m1.696s

Check it worked. Both runs end in no errors were found. Honest expectation-setting on the timings: my repository is 2 MB, so under two seconds. --read-data reads everything. A 100 GB repository means reading 100 GB — over the network for the off-site one. Run this one quarterly, or use --read-data-subset 10% monthly to spread the cost.

Something went wrong?

• You see Permission denied (publickey,password) when restic talks to the remote → it means SSH key login is not in place for that host → redo Step 1's key part; restic cannot type passwords for you.
• You see a literal $USER inside the service file when you cat it → it means the heredoc was quoted or copied oddly, so the shell never expanded it → remove the appended line and re-paste the Step 2 block from the first line to EOF.

Undo all of this

Tested for real, in this order — I tested it as part of the combined undo with the timers guide, whose undo deletes the whole service file. Deleting that file also removes the off-site line this guide appended in Step 2.

Remove the drill script, the off-site repository and the scratch folder:

sudo rm /usr/local/bin/restore-drill

rm -rf ~/offsite /tmp/restore-drill

If you used the localhost stand-in, remove the self-trust key too (skip if you used a real remote — just delete the key from that machine's authorized_keys instead):

sed -i "/$USER@$(hostname)/d" ~/.ssh/authorized_keys

Only do this if you created the key for this guide. If Step 1 found an existing key and you reused it, skip this — deleting it would lock you out of every server that trusts it. Check first: a key you made today shows today's date with ls -l ~/.ssh/id_ed25519.

rm ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.pub

When I tested this undo, key login to localhost was refused again (Permission denied (publickey,password)) — the machine no longer trusts itself. Then run the timers guide's undo to remove the service and timer, and the restic guide's undo to remove restic itself. I ran all of them back-to-back in that order and ended with a completely clean machine.

Where to go next

• Know when things break: a tiny monitoring setup — so a failed nightly backup pings you instead of rotting silently.
• Backups while you sleep: systemd timers and retention — the first half of this pair, if you landed here directly.
• Official docs: restic — scripting and automation.

---

Last tested: 13 August 2026 on Ubuntu 24.04.4 LTS. Versions: restic 0.16.4 (Ubuntu archive), systemd 255. Off-site leg tested against localhost sshd as an honest stand-in for a real remote; transcripts live in ../guide-15-backup-automation/.