In a makefile, I have several rules that look like this:
out.txt: foo.sh input.txt ./foo.sh -i input.txt > out.txt If foo.sh fails, then out.txt will be created as a 0-sized file. If I run make again, it will mistakenly assume that the out.txt file was created successfully, and it won't run the rule again.
What's the right way to handle this sort of scenario?
You can request that make delete the target file if the rule fails, by defining a special target named .DELETE_ON_ERROR. It doesn't need to do anything or have any dependencies, so just add this to your makefile:
.DELETE_ON_ERROR: Then you get the following:
$ cat Makefile .DELETE_ON_ERROR: foo: false > foo $ make false > foo make: *** [foo] Error 1 make: *** Deleting file `foo' zsh: exit 2 make $ stat foo stat: cannot stat `foo': No such file or directory From Errors in Recipes:
Usually when a recipe line fails, if it has changed the target file at all, the file is corrupted and cannot be used—or at least it is not completely updated. Yet the file's time stamp says that it is now up to date, so the next time
makeruns, it will not try to update that file. The situation is just the same as when the shell is killed by a signal; see Interrupts. So generally the right thing to do is to delete the target file if the recipe fails after beginning to change the file.makewill do this if.DELETE_ON_ERRORappears as a target. This is almost always what you wantmaketo do, but it is not historical practice; so for compatibility, you must explicitly request it.
.DELETE_ON_ERROR is that it automatically applies to every rule. It has downsides though: it's specific to GNU make; it doesn't trigger if the build is interrupted violently (power failure, unmaskable signal); it removes the partial file, which can be useful in investigating failed builds. Whenever possible, makefile rules should first create their target under a temporary name, then move it into place. That way, if the build process is interrupted for any reason, there won't be a half-written target file that can't be distinguished from a fully written file.
out.txt: foo.sh input.txt ./foo.sh -i input.txt >[email protected] mv -f [email protected] $@ mv -f [email protected] $@ is a common makefile idiom.
Juliano's answer shows a variant where the name of the temporary file is generated dynamically. The dynamic name generation is required if there may be more than one process generating the same target or if the directory can be written to by other users. This is very rarely the case for a build tree (if these were issues, a lot of what's going on in a typical makefile would break), so the extra complexity is usually not necessary.
.DELETE_ON_ERROR is in my answer; with that on there should be no need for this temp file juggling make dies without having time to clean up (power failure, OOM, user sending a KILL signal, …)? kill -9ing make you deserve what you get, and I would hope somebody is going to do a clean build if there's something exceptional like a power failure Since it seems that foo.sh is something you have written (or is generated by something you wrote), I would consider adding a -o flag to it, which takes the output file name. Then you do not have to depend on the default I/O redirection behavior. Your script can clean up partial output files, or possibly even avoid creating the file in the first place if you can detect the error early enough.
I suppose that foo.sh properly returns non-zero on error. Then you should make a temporary, and only overwrite the output on success.
tmp=$$(mktemp) && ./foo.sh input.txt > $$tmp && mv $$tmp $@ || rm -f $$tmp && false 
$tmp if the generation fails, partial files can be useful to debug failed builds. If you do, you must make sure the command fails if rm is invoked, e.g. || rm $$tmp && false.