0

I'm trying to copy a bunch of files (around 50000) from an include-file list. THe include-file has relative path to the files on the source folder. I'm using the following command :

rsync -avPilHAXWE \ --include="*/" \ --include-from=/tmp/mover-include.list \ --exclude="*" \ --prune-empty-dirs \ --remove-source-files \ /mnt/source/ /mnt/dest/

The include file has multiple files that are hard linked, for example in the file :

dir1/dir2/dir3/filea .... .... dir1/dir5/fileb

rsync correctly copies filea, but when arriving to fileb, it doesn't copy it (so I think he still considered it a hardlink, but neither did it recreate the hardlinks. So filea end ups alone on /mnt/dest, with no hard links, and fileb ends up alone on /mnt/source/ with no hardlinks.

This shouldn't change anything, but my dest folder is on a fusefs merging multiple disks

Anybody have an idea on how to solve this? Thanks

Improve this question
11
  • --include="*/" --include-from=/tmp/mover-include.list that seems contradictory? Commented Oct 21 at 11:55
  • Not at all, the first --include="*/" is mandatory to include root folders Commented Oct 21 at 11:56
  • my rsync filter rules are a bit rusty, but why would you need to do that (you would need to do that for each lower directory containing one of the files) – you only want the files from the list, right? wouldn't -R solve that elegantly? Commented Oct 21 at 11:59
  • I didn't try it. Possibly, but that certainly doesn't answer my question. And I don't really see the relevance of the -R (which tells rsync to use relative paths) Commented Oct 21 at 12:05
  • 1
    hm, the point is that we don't have access to your full list, nor to your actual file set. Chances of rsync bugs are really far from zero, but in this case my bet is the old adage that "software does exactly what it's told to; that's what makes it so harmful" applies: It simply seems fileb simply does not get included for some reason. So, simplifying your rsync command line to isolate exactly what leads to that file not being included is not only an exercise in elegance, it's a necessity to be able to reproduce or understand the problem, and in the end, solve it :) Commented Oct 21 at 12:48

1 Answer 1

Reset to default
4

OK, the problem was simple. --includes-from= is waiting for regexp from files to import. My file didn't have regexp but file names, sometimes with spaces or other special character, making them ignored by --includes-from.

The very simple solution is to use --files-from= and droping the others include and exlude.

Improve this answer
1
  • glad to read you got it sorted out :) Commented Oct 21 at 15:38

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.