Timeline for rsync with --include-from breaks hardlinks (evec with -H)
Current License: CC BY-SA 4.0
16 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 21 at 14:22 | answer | added | François-Xavier Payet | timeline score: 4 | |
| Oct 21 at 12:48 | comment | added | Marcus Müller | hm, the point is that we don't have access to your full list, nor to your actual file set. Chances of rsync bugs are really far from zero, but in this case my bet is the old adage that "software does exactly what it's told to; that's what makes it so harmful" applies: It simply seems fileb simply does not get included for some reason. So, simplifying your rsync command line to isolate exactly what leads to that file not being included is not only an exercise in elegance, it's a necessity to be able to reproduce or understand the problem, and in the end, solve it :) | |
| Oct 21 at 12:18 | history | edited | François-Xavier Payet | CC BY-SA 4.0 | deleted 28 characters in body |
| Oct 21 at 12:17 | comment | added | François-Xavier Payet | Sorry, but that's absolutely not the point of this conversation. I've got an rsync command that should preserve hardlinks, I'd like to understand why it doesn't, that's all. | |
| Oct 21 at 12:15 | comment | added | Marcus Müller | yes, but as said, that job can be done with less convoluted shell scripting! Knowing that this is the main reason for using rsync here is helpful :) | |
| Oct 21 at 12:12 | comment | added | François-Xavier Payet | mv certainly won't preserve my hardlinks. Preserving my hardlinks is the main reason for using rsync | |
| Oct 21 at 12:10 | comment | added | Marcus Müller | With -W you disable delta transfers, i.e., are you really well-served using rsync? This feels like a job for xargs mv < mover-include.list or parallel mv :::: mover-include.list. Hardlinks can't be preserved across mounts that way, but even that could be achieved with relatively little effort through shell scripting means (or a proper programming language) to preprocess the list and generate a "afterwards, create these hardlinks" step | |
| Oct 21 at 12:10 | comment | added | François-Xavier Payet | I've tried with my minimum : -avAHXW \ --progress \ --stats \ --human-readable \ --prune-empty-dirs \ --include="/" \ --include-from="/tmp/mover-include.list" \ --exclude="" A?d have the exact same issue | |
| Oct 21 at 12:05 | comment | added | Marcus Müller | another thing, when I see avPilHAXWE, I'm almost certain that this is from a "much helps much" appoach, and if any of these flags has any side effects on including files, we're making our job unnecessarily hard. Since this is a local copy, you ideally have no need for -P (no partial transfers, none at all if things can be mved on the same file system), -l is included in -a already, E is included in -a; but the honest question here is this: | |
| Oct 21 at 12:05 | comment | added | François-Xavier Payet | I didn't try it. Possibly, but that certainly doesn't answer my question. And I don't really see the relevance of the -R (which tells rsync to use relative paths) | |
| Oct 21 at 12:01 | history | edited | Marcus Müller | CC BY-SA 4.0 | added 63 characters in body |
| Oct 21 at 11:59 | comment | added | Marcus Müller | my rsync filter rules are a bit rusty, but why would you need to do that (you would need to do that for each lower directory containing one of the files) – you only want the files from the list, right? wouldn't -R solve that elegantly? | |
| Oct 21 at 11:56 | comment | added | François-Xavier Payet | Not at all, the first --include="*/" is mandatory to include root folders | |
| Oct 21 at 11:56 | history | edited | François-Xavier Payet | CC BY-SA 4.0 | edited body |
| Oct 21 at 11:55 | comment | added | Marcus Müller | --include="*/" --include-from=/tmp/mover-include.list that seems contradictory? | |
| Oct 21 at 11:55 | history | asked | François-Xavier Payet | CC BY-SA 4.0 |