1

I am trying to make a copy of completed files that are saved to a watch folder. I have created a bash script using inotifywait to watch the folder.

My original script worked great with files and folders/subfolders. Once I replaced the "-e create -e moved_to" with "-e close_write" it no longer recognizes/processes folders that are saved in the watch folder. It only works now with files. The reason I am needing to change to this is because a lot of my files are large (and multiples in folders) and takes some time to be saved completely (to the watch folder) and the script runs through all the processes before the files are completely saved/copied.

I have also tried the "wait" command but it does nothing at all. I know there are more complicated/better/correct ways of using it but I have not figured them out yet. After finding the "close_write" event in inotify I felt it would do exactly what I am needing by waiting to for the current process/command to finish before starting the next. And it works perfectly on large single files but is MIA (does nothing) whenever a folder is copied to the watch folder.

Here is a sample of my script. I am not well versed in writing bash scripts so sorry if it is a mess.

#!/bin/bash WATCHED=/mnt/Watched DESTINATION=/mnt/Destination user=userid group=groupid perms=777 # Step 1 - Copying completed files to Destination folder inotifywait -m -e close_write --format %f $WATCHED \ | while read new do echo Detected new file $new, Copying to Destination folder cp -r "$WATCHED/$new" "$DESTINATION/" wait # Step 2 - Deleting unwanted files from Destination folder echo Deleting unwanted files from Destination folder find $DESTINATION -type f -iname "*.txt" -delete find $DESTINATION -type f -iname "*.nfo" -delete find $DESTINATION -type f -iname "*.website" -delete find $DESTINATION -type f -iname "*.exe" -delete find $DESTINATION -type f -iname "*.html" -delete find $DESTINATION -type f -iname "*.htm" -delete find $DESTINATION -type f -iname "*.sfv" -delete find $DESTINATION -type f -iname "*.parts" -delete find $DESTINATION -type f -iname "*.jpg" -delete find $DESTINATION -type f -iname "*.png" -delete find $DESTINATION -type f -iname "*.doc" -delete sleep 10 # Step 3 - Change permissions of new files in Destination folder echo Changing ownership and permissions to Destination folder and files chown -R $user:$group "$DESTINATION" chmod -R $perms "$DESTINATION" done

Any guidance would be appreciated.

Thanks

EDIT...

I cannot for the life of me get this script to even see a new folder added to the watch folder. It literally does NOTHING. If I replace the close_write with create & moved_to and make no other changes it sees the folder and contents and processes correctly. I find this very odd.

In fact I even tried to make a small test script to see if I can get it to work with a if/elif/else statement but once again when I copy a folder in the watch folder the script does nothing (doesn't even make to the loop). If I put a file in then it provides the correct output.

Here is the test script I ran. Can someone else confirm if they can get a new folder to be recognized and processed correctly?

#!/bin/bash WATCHED=/mnt/Watched inotifywait -re close_write --format '%w%f' -m $STEP1_WATCHED \ | while read -r new do if [[ -d "$new" ]]; then echo "Detected new folder $new" elif [[ -f "$new" ]]; then echo "Detected new file $new" else echo "neither $new" fi done done
Improve this question
2
  • That's not hard to find out, by some testing, to see what events you need to monitor. Just print them while reproducing the cases. I guess you want close_write and moved_to ? Commented Dec 15, 2020 at 19:40
  • I have been testing this for days on end although I have not tried print yet as I am not sure of the correct syntax. I have tried to run both create as well as moved_to with close_write but then close_write does not work because the others are true. Meaning it will just start running the script without waiting for the files to finish copying first. Thanks for the help! Commented Dec 15, 2020 at 19:51

1 Answer 1

Reset to default
3

The reason you're not seeing directories created from your script is that you're looking for CLOSE_WRITE instead of CREATE with IS_DIR:

inotifywait . # Terminal 1 Setting up watches. Watches established. ./ CREATE,ISDIR zz mkdir zz # Terminal 2

Now, because CREATE can also apply to files you need to test the type of item as part of your decision making:

#!/bin/bash watchDir=/mnt/watched inotifywait -qre CLOSE_WRITE,CREATE --format $'%e\t%w%f' --monitor "$watchDir" | while IFS=$'\t' read -r events item do if [[ -d "$item" ]] then echo "New directory '$item'" elif [[ "$events" =~ CLOSE_WRITE ]] then if [[ -f "$item" ]] then echo "New file '$item'" elif [[ -e "$item" ]] then echo "New unknown item '$item'" fi fi done

This code will fail for file or directory names that contain newlines (touch $'break\nfile') or start with one or more tab characters (touch $'\ttabbed'), as inotifywait cannot use a format string that contains a null character (\000). There is also a small race condition where the event can fire but the target is potentially removed or even replaced before the loop can be processed and the target type evaluated.

Improve this answer
3
  • roaima, Please see my reply to your comment above. I replied in the wrong section :-/ Commented Dec 16, 2020 at 1:27
  • 1
    @S.Ray I'm not entirely sure why you wanted to catch directory creation - if it's so that file copies have a structure to be written to, then a tool like rsync will create the intervening directories as part of a file copy and your original code will (mostly) work just as is. Commented Dec 16, 2020 at 9:58
  • roaima, I ended up changing my script completely and using rsync to copy my folders/files. Thanks! Commented Dec 21, 2020 at 3:33

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.