How do I automatically do daily backups and on every shutdown, and restore them elsewhere daily and on boot?

I'd go about this as follows:

1. install restic on both primary and secondary. Assuming you're on a non-esoteric Linux distro, Mac OS, Free- or OpenBSD, that's one {your package manager} install restic away, see instructions.
2. (Set up SSH public key authentication for the user root for the store server as described above, i.e., run sudo -i -H ssh-keygen and sudo -H -i ssh-copy-id [email protected] on both primary and secondary.)
3. generate a secure password in a file only readable by root. On the primary, machine run, sudo sh -c 'touch /etc/restic-backup-pass && chmod 600 /etc/restic-backup-pass && head -c 128 /dev/random > /etc/restic-backup-pass'.
4. Copy that file to the same location on the secondary. Make sure it's still only readable by root.
5. Initialize the repository. On primary, run

sudo restic --password-file /etc/restic-backup-pass --repo "sftp://[email protected]:restic" init 

6. Run the first backup: on the primary, do

sudo restic \ --password-file /etc/restic-backup-pass \ --repo "sftp://[email protected]:restic" \ backup \ /home 

7. Get the backup: on the secondary, run

sudo restic \ --password-file /etc/restic-backup-pass \ --repo "sftp://[email protected]:restic" \ restore \ latest \ --target /home 

When 1. – 7. work, well, time for automation. On the primary, you create a service that does the backup, on the secondary, you create a service that restores the backup. You run the service on the primary a) on every shutdown, and b) every day at noon, and you run the service on the secondary a) on every boot before nginx starts and b) every day at 13h. (and of course, you can start it easily manually).

To set up the backup service: on primary,

8. run (I'm assuming nvim is your favourite text editor. If not, replace it with something else, or omit the EDITOR=nvim altogether to use the default editor):

sudo env EDITOR=nvim systemctl --force --full edit backup-to-store.service 

and put in something like

[Unit] Description="Backing up /home to store" Wants=network.target After=network.target [Service] ExecStart=/usr/bin/restic --password-file /etc/restic-backup-pass --repo "sftp://[email protected]:restic" backup /home Type=oneshot [Install] WantedBy=shutdown.target 

and save and exit the editor. 9. Test that service: sudo systemd-analyze verify /etc/systemd/system/backup-*. If that's OK, sudo systemctl start backup-to-store.service, followed by journalctl -xef. You should see the output of restic in that system log! 10. Enable the service to be automatically run at shutdown: sudo systemctl enable backup-to-store.service 11. Add a timer that runs the service Monday through Saturday at noon (note the .timer):

sudo env EDITOR=nvim systemctl --force --full edit backup-to-store.timer 

contents would be:

[Unit] Description="Run workday backup" [Timer] OnCalendar=Mon..Sat *-*-* 12:00:* Unit=backup-to-store.service [Install] WantedBy=multi-user.target 

and verify, sudo systemd-analyze verify /etc/systemd/system/backup-*, and then enable the timer:

sudo systemctl enable backup-to-store.timer 

Great! Now we have automated, and incremental backups with passwords and authentication keys unreadable to normal users.

Time to work on restoring. On secondary,

12. run

sudo env EDITOR=nvim systemctl --force --full edit get-backup-from-store.service 

with contents something like

[Unit] Description="Restoring /home from store" Wants=network.target After=network.target Before=nginx.service [Service] ExecStart=restic --password-file /etc/restic-backup-pass --repo "sftp://[email protected]:restic" restore latest --target /home Type=oneshot [Install] WantedBy=multi-user.target 

verify and enable: sudo systemd-analyze verify /etc/systemd/system/get-backup-*, sudo systemctl enable get-backup-from-store.service 13. Set up the restore timer:

sudo env EDITOR=nvim systemctl --force --full edit restore-from-store.timer 

with content

[Unit] Description="Get workday backup" [Timer] OnCalendar=Mon..Sat *-*-* 13:00:* Unit=backup-to-store.service [Install] WantedBy=multi-user.target 

and verify, sudo systemd-analyze verify /etc/systemd/system/get-backup-*, and then enable the timer, sudo systemctl enable get-backup-from-store.timer.

Great, now we have automated, password-safe, incremental, backup restore on the secondary.

You can run a backup manually by starting the service you created manually on the primary (sudo systemctl start backup-to-store.service) and you can download the latest manually by running the service on the secondary manually (sudo systemctl start get-backup-from-store.service).

What's missing here is the clean up older snapshots (for example, the daily snapshots that are older than 2 weeks can probably be dropped, but keeping a monthly backup might be nice). For details on that, refer to restic's documentation on "Removing snapshots according to a policy", but in short, you'd want to run

sudo restic \ --password-file /etc/restic-backup-pass \ --repo "sftp://[email protected]:restic" \ forget \ --keep-daily 14 \ --keep-monthly 4 \ --keep-yearly 5 

or such, every 14 days (hey, that's another service and timer for primary).