0

Two Part Question

PART ONE: I want to use the the following rsync command to back up my Ubuntu 18.04 file system starting at / during an Ubuntu Live Session (meaning my file sys), but I am not sure if I all the necessary / recommended excludes and I am not sure if my syntax is correct or not:

sudo rsync -aAXHv --delete --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/home/*","/swapfile"} /from/live/session/to/my/file/system/ /path/to/my/external/storage/file-system-18.04/

I plan to re-use this command to back up my file system periodically to the same external hd location, so I include --delete

Specifically, under /root/ directory I have .cache, .config, .dbus, .gnupg, .gvfs, .local, .mozilla

I know that gvfs, mozilla, dbus, and gnupg -must- be excluded (if I am wrong let me know), but how would the excludes look? (I also want to exclude the local/share/Trash folder as well)

sudo rsync -aAXHv --delete --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/home/*","/swapfile","/root/*/gvfs/","/root/*mozilla/","/root/*/dbus/","/root/*/gnupg/","/root/*/local/share/Trash"} /from/live/session/to/my/file/system/ /path/to/my/external/storage/file-system-18.04/

Is the syntax above right? What am I missing?

PART TWO: After I backup my file system I want to automatically save it to a zip archive to be burned to a disk, uploaded, etc. (good back systems are redundantly paranoid) I want to use this command:

sudo zip -p password -ry "/path/to/my/external/storage/post-rsync-zips/file-system-18.04-backup.$(date +"%F").zip" /path/to/my/external/storage/file-system-18.04

During testing I start a zip warning: name not matched message, which I expect because the actual archive has not yet been created at /path/to/my/external/storage/post-rsync-zips/ .

However, here's the odd ball problem: the zipping process actually quits with the following error:

zip I/O error: No space left on device zip error: Output file write failure (write error on zip file)

I am using a 2TB external the file-system-18.04 folder is 230GB and another folder of previous backups of 430GB, leaving 1280GB of free space for my zip directory to use.... so why am I getting the error?

Improve this question

1 Answer 1

Reset to default
2

At the risk of giving a somewhat off-topic/alternative answer: do yourself a favour and use a tool like borgbackup for this purpose. It seems to tick all your boxes and will be

  • easier to use
  • easier to maintain (incl. your inclusions/exclusions of your backup set)
  • save more space (de-duplication and compression included)
  • give you the added benefit of:
    • encryption
    • redundant history (à la time machine)

I'm not afilliated with borgbackup but a happy user. More info here.

EDIT: at the request to still answer the question "more directly" here a few ideas:

Use the exclude flag as explaned in the man page, i.e. with a pattern as argument like --exclude='*.log', you can use it multiple times for different patterns. Be careful with overlapping includes/excludes, they may have a different result than expected (I've had issues with this before). If your patterns are many and/or complex, put them in a text file and refer with --exclude-from=excludes.txt

For the compression part: I'm not sure about zip and the cause of your issue (more info would be needed for that), it might have to do with an intermediate file being made in the background, or an "obsolete" filesystem with e.g. a 2GB file size limit.

However, for linux you might be better off with modern compression which is supported out of the box with tar, like xzor bz2 (the former being quite efficient in comparison), also because you can just pipe it somewhere during generation, so guaranteed without intermediate files.

But again, you're missing out on very attractive functionality by not using a more specialised tool, especially because you are compressing after rsync, which makes incremental backups of your compressed archives impossible. I'd argue KISS is nice, but KISS is much better (second S=Smart) and there are ubiquitous, open source tools out there that are just that: smarter than "just" rsync alone.

Cheers, ppenguin

Improve this answer
9
  • If you need a technical reason why using rsync to backup is advantageous you should read this. But having used Ubuntu for X yrs, watched X number of Youtube videos, and read X number of user reviews for the latest/greatest backup systems. I have still been burned at X times on corrupted backups that wouldn't restore or 3rd party apps whose code dependencies were deprecated, or whose restore system was convoluted and no support. Commented Jul 12, 2020 at 2:53
  • So now I live by the KISS philosophy. I write a 2 command shell script, that backs up to an encrypted external hard drive, so no worry about compression/archive/file corruptions. And the commands used rsync and zip are so universal in the unix world that I never worry about deprecation or updates. Once I figure out my excludes and the whacky zip error, which I think is being caused by symlinks, I only need to teach myself how to run my shell script automatically using a cron job to keep my folder backed-up regularly. Commented Jul 12, 2020 at 3:02
  • As for my file system I have no problem booting once a month or after major changes to run my rsync command. Why? Because if my primary system or machine ever fails I know that I can take my 1,000% reliable back up to any machine install ubuntu, run rsync in reverse and be back up and running within 2 hours... easy peasy. Commented Jul 12, 2020 at 3:02
  • 1
    If someone can answer my questions, you can save me some testing time... :-) Commented Jul 12, 2020 at 3:06
  • @yupthatguy, don't get me wrong, I also live by KISS and have used rsync extensively, but still I think that borgbackup would be better suited for this purpose. But I might have an idea (just a guess for now) for your specific issue, see the edit above... Commented Jul 13, 2020 at 13:16

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.