2

I want to be able to sync files from multiple directories to one directory. If I remove a directory from the source list, then I want that directory and all of its files to be removed from the destination.

Here's a quick sample:

mkdir dir1 touch dir1/file1 mkdir dir2 touch dir2/file2 mkdir dir3 rsync -a dir1 dir2 dir3

dir3 now contains "dir1/file1" and "dir2/file2".

If I rsync again, and remove dir2 from the list:

rsync -a dir1 dir3

Then I would like dir2/file2 and dir2 to be removed from the destination.

The --delete option to rsync only handles deletes between dir1 and dir3/dir1, so dir2 remains (even if using --delete).

I've looked through the various option in the man page for rsync, and I don't see any option that tells rsync to "combine" all the sources and consider them as "one" tree so that "--delete" would do what I want it to do. However, I'm pretty confident I'm not the first person to want to do something like this.

Ideas?

Improve this question
2
  • I think it's not possible using the syntax that you want. I think you nail it with your last paragraph. rsync doesn't "know" about dir2, so it won't delete it. Commented May 28, 2019 at 0:47
  • It's just not clear to me why rsync doesn't have an option which makes it consider all the sources as one master source. That would solve the problem. However, that may be addressed below by Freddy... Commented May 28, 2019 at 2:15

1 Answer 1

Reset to default
2

The --delete flag obviously doesn't work if you sync multiple source directories to one destination directory. Instead of syncing dir1 and dir2 to dir3 you could sync the content of the parent dir of dir1 and dir2.

Example (using rsync flag -i to see the changes):

$ mkdir -p dir12/dir1 dir12/dir2 dir3 $ touch dir12/dir1/file1 dir12/dir2/file2 $ rsync -ai dir12/ dir3 cd+++++++++ dir1/ >f+++++++++ dir1/file1 cd+++++++++ dir2/ >f+++++++++ dir2/file2 # BEFORE: dir1 and dir2 were synced to dir3 $ rm -rf dir12/dir2 $ rsync -ai --delete dir12/ dir3 *deleting dir2/file2 *deleting dir2/ .d..t...... ./ # AFTER: dir3/dir2 is gone

If you don't have a common parent directory, then dir12/dir2 could be a symlink pointing to the real directory. With option -L (--copy-links) in the first rsync call you would get the same directory structure in dir3 with dir2 as directory (instead of a symlink). And after deleting symlink dir12/dir2 and syncing again with --delete the directory dir3/dir2 would be gone like in the example.

EDIT:

I fiddled a bit around and may have found a hack:

$ mkdir empty dir1 dir2 dir3 $ touch dir1/file1 dir2/file2 $ rsync -ai dir1 dir2 dir3 cd+++++++++ dir1/ >f+++++++++ dir1/file1 cd+++++++++ dir2/ >f+++++++++ dir2/file2 $ rsync -ai --delete empty/ dir1 dir3 *deleting dir2/file2 *deleting dir2/ .d..t...... ./

Use empty/ as a placeholder for all directories you want to delete.

Improve this answer
6
  • The question is unclear, but I assumed that dir1 and dir2 were not necessarily in the same directory. I thought about -L, but that potentially conflicts with -a from the question. Commented May 28, 2019 at 1:19
  • Yes, that's why I thought he could create a (fake) parent directory and use symlinks instead. For the example with symlink dir12/dir2 I used rsync -aiL dir12/ dir3 and that seems to work, but I haven't done any further testing with more complex setups. But... if OP knows that dir2 will be excluded from the next sync, it might be easier to manually rm -rf the directory on the destination side. Commented May 28, 2019 at 1:30
  • Yeah manually rm makes more sense to me. They have to change the rsync command anyway. Commented May 28, 2019 at 2:04
  • I think there's some promise to using -L and I will experiment with that tomorrow. Basically, I have a tool that is using rsync. One user specifies the directories that they want synced when the tool is run. A different user runs the tool that actually does the syn. The initial user can change the directory lists to be synced at any time. However, the syncing user should see the "end" result - a perfect copy with no "extra" directories. If I build a directory containing links to all the directories first, this would work. I just wish I didn't need that extra step. Commented May 28, 2019 at 2:22
  • As it happens, I don't think -L will work after all. After the rsync completes, the system ensures that all synced files are mode 644 and directories 755. This would mean that someone could symlink to a file they couldn't read, then after the sync, because of -L and the permission 'fixing' they would have access to the file. I'll have to think this through some more. Commented May 28, 2019 at 3:04

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.