4

I'm trying to create a backup script. I've managed to get this script working fine on a CentOS 6.7 machine and am now trying to get it working properly on Debian 7.

I am running into a problem I can't seem to solve with Google or any of the information found on this site. I'll try to explain my situation before getting into the problem.

On CentOS, I use the following command to find files that have been changed in the past 24 hours in $SOURCEDIR and use xargs to put only these files into $ARCHIVE. If no files are found a message pops up. 

find $SOURCEDIR -mtime -1 -print | xargs -r tar rcvf $ARCHIVE || { echo "No files have been changed in the past 24 hours. Exiting script ..." ; exit 1; }

I am aware that using tar rcvf can invoke the following error message:

You may not specify more than one '-Acdtrux' or '--test-label' option

This however, does not seem to happen on the CentOS machine. It does on the Debian machine, thus I've removed the r command from the tar command. The reason I've added this in the first place is because I want to avoid the archive being overwritten if find would return more than 100 results.

Now onto the actual problem. Whenever I run

find $SOURCEDIR -mtime -1 -print

I get a list of the files that have been changed in $SOURCEDIR in the past 24 hours, as expected. However, whenever I run the complete command including the pipe symbol and the xargs command like this:

find $SOURCEDIR -mtime -1 -print | xargs -r tar cvf $ARCHIVE || { echo "No files have been changed in the past 24 hours. Exiting script ..." ; exit 1; }

I actually see the find command print all files from $SOURCEDIR before I end up with an archive including all the files from $SOURCEDIR, and I do not understand why. Any help would be greatly appreciated.

Improve this question
8
  • does the output of find $SOURCEDIR -mtime -1 -print also include . as one of the output? Commented May 12, 2016 at 13:44
  • No, there is no . included in the output of the find command. The output includes the name of the $SOURCEDIR and all the files that have been changed in the past 24 hours. Commented May 12, 2016 at 13:48
  • and I guess $SOURCEDIR doesn't show up on CentOS? you just need to remove that before piping to tar.. Commented May 12, 2016 at 13:52
  • I get the exact same result on CentOS, the $SOURCEDIR is the first result. I've edited the original post to include the fact that I get to see all files listed by find when I run the full command. Commented May 12, 2016 at 13:57
  • 4
    Give find a -type f argument if you only want it to print pathnames of files. Otherwise, it'll print directory pathnames, too. Commented May 12, 2016 at 14:21

4 Answers 4

Reset to default
2

As others have identified, the problem with your command is that it includes directories, and tar archives them recursively. If a directory has been modified recently, all the files in it and its subdirectories get included, whether they have been modified or not.

If you don't care to back up directory metadata, then just tell find not to print directory names. It isn't enough to omit the root: the same thing can happen with subdirectories too.

find "$SOURCEDIR" -mtime -1 ! -type d -print | xargs -r tar -rcf "$ARCHIVE"

Using xargs fails with file names containing spaces and some other special characters. This is easy to fix: use -exec instead of xargs.

find "$SOURCEDIR" -mtime -1 ! -type d -exec tar -rcf "$ARCHIVE" {} +

If you want to back up directory metadata, let find print everything and instead tell tar not to recurse into subdirectories. Since find is doing the recursion, tar doesn't need to.

find "$SOURCEDIR" -mtime -1 -exec tar -rcf "$ARCHIVE" --no-recursion {} +

With this approach, you can avoid the use of tar -rc and instead solve the problem of repeated tar invocations by first creating an archive with only the root directory, and then appending to it in batches. (Why the root directory? Because GNU tar is afraid of creating an empty archive.)

tar -cf "$ARCHIVE" --no-recursion "$SOURCEDIR" find "$SOURCEDIR" -mindepth 1 -mtime -1 -exec tar -rf "$ARCHIVE" --no-recursion {} +
Improve this answer
3
  • very well explained with incremental thought process, so much to learn here Commented May 13, 2016 at 2:15
  • Thanks for the detailed explanation. But what does the {} + at the end of the commands do? Commented May 13, 2016 at 8:11
  • @Seroczynski It's the syntax of the -exec action of find. {} is replaced by the list of file names, as many as fit in the command line length limit. For example -exec command arg1 arg2 {} + executes command arg1 arg2 file1 file2 … fileN1 then command arg1 arg2 fileN1+1 … fileN1+N2 and so on. There's also -exec … {} … \; which executes the command for each file, one at a time. Commented May 13, 2016 at 11:41
2 
find $SOURCEDIR -mtime -1

also includes $SOURCEDIR in the output, which needs to be removed before further processing

Using grep -vx one can define a particular line to be excluded..

find $SOURCEDIR -mtime -1 -print | grep -xv "$SOURCEDIR" | xargs -r tar cvf $ARCHIVE || { echo "No files have been changed in the past 24 hours. Exiting script ..." ; exit 1; }
Improve this answer
1
  • 1
    Simpler: find "$SOURCEDIR" -mindepth 1 -mtime -1. But going further, the same problem will occur with subdirectories, so -type f is probably the right solution here (unless directory metadata matters). Commented May 12, 2016 at 22:10
2

You will not only run into problems if xargs invokes tar twice, you also will get problems if your file-names contain special characters like newlines.

You should drop the use of xargs and tar and use find with cpio:

find $SOURCEDIR -mtime -1 -print0 | cpio --create -0 --verbose \ --format=ustar -O $ARCHIVE

ustar provides you with a POSIX.1 compliant tar file in $ARCHIVE.

Improve this answer
1
  • Thanks, I'll look into this. This is the reason I've been using rcvf in CentOS (as mentioned). Commented May 12, 2016 at 14:23
0

When you execute find $SOURCEDIR -mtime -1 -print it will have as the first result the folder $SOURCEDIR itself. That is why everything is included.
You have to exclude the first result or $SOURCEDIR.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.