123

How can a program monitor a whole directory tree for changes in Linux (ext3 file system)?

Currently the directory contains about half a million files in about 3,000 subdirectories, organized in three directory levels.

Those are mostly small files (< 1kb, some few up to 100 kb). It's a sort of queue and I need to know when files are being created, deleted or their content modified within 5-10 seconds of that happening.

I know about inotify and other similar APIs, but AFAIK they only monitor a single directory, which means I would need 3,000 inotify handles in my case - more than the usual 1024 handles allowed for a single process. Or am I wrong?

In case there isn't a Linux API that can tell me what I need: perhaps there is a FUSE project that simulates a file system (replicating all file accesses on a real file system) and separately logs all modifications (couldn't fine one)?

Improve this question

12 Answers 12

Reset to default
111

I've done something similar using the inotifywait tool:

#!/bin/bash while true; do inotifywait -e modify,create,delete -r /path/to/your/dir && \ <some command to execute when a file event is recorded> done

This will setup recursive directory watches on the entire tree and allow you to execute a command when something changes. If you just want to view the changes, you can add the -m flag to put it into monitor mode.

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

To avoid the while loopuse the -m or --monitor switch/option/flag/arg. Don't know when that 'switch' came into being, but is better than loops
You should also add the move event: inotifywait -e modify,create,delete,move -r /path/to/your/dir
Couldn't this approach miss an event in case when two of them happen in a split second? After inotifywait exists there would be a period when no events are monitored, wouldn't it?
@gwillie, but if m flag is used, it just outputs to stdout and we cannot execute any command using that trigger. So, if you want to execute something after any event is observed, isn't the while loop better?
inotifywait -e modify -m | while read line; do something_with $line; done
58 
$ inotifywait -m -r /path/to/your/directory

This command is enough to watch the directory recursively for all events such as access, open, create, delete ...

Improve this answer

1 Comment

Yes but events such as access and open are very problematic. Depending of what your intent is. Example: I wanted to relaunch cordova run each time something change in www directory. As result, the open,access events generated by cordova was triggering inotifywait, entering in an infinite loop. -e modify,create,delete,move is better for most uses.
33

To my knowledge, there's no other way than recursively setting an inotify watch on each directory.

That said, you won't run out of file descriptors because inotify does not have to reserve an fd to watch a file or a directory (its predecessor, dnotify, did suffer from this limitation). inotify uses "watch descriptors" instead.

According to the documentation for inotifywatch, the default limit is 8192 watch descriptors, and you can increase it by writing the new value to /proc/sys/fs/inotify/max_user_watches.

Improve this answer

5 Comments

Sounds good. Any negative aspects to consider when using so many watch descriptors?
Nope, apart from the time taken to create all the watches, I don't think you'll run into issues with only 3000 subdirectories.
Does this not create possible race issues like: creation of folder_sub in folder_main, creation folder_sub_sub in folder_sub, inotify of folder_main arrives, watch is set on folder_sub, but folder_sub_sub is already missed, and thus there is furthermore no watch installed on it?
Ubuntu 18.04 now has the default of 'max_user_watches' now set to 65536 which seem to be a reasonable values in normal desktop/server systems.
@KoenG. yes, that's one possibility of losing some events, but you can have "eventual consistency", i.e. you'll know "1 or more changes happened here" since your last in-memory picture. You then install a watch on main/folder_sub/folder_sub_sub and read its current files. By that time, it's possible that some file was created, changed and deleted and you'll not even know it. Consider inotify as optimized "trigger to re-scan", not "full history"; if you must have full history go for something like FUSE or strace.
24

inotify is the best option when you have many subdirectories but if not I am used to using this command below:

watch -d find <<path>>

Improve this answer

4 Comments

watch is def preferred
watch doesn't allow paging, so it'll lose anything that is longer than the terminal height (e.g., tree commands with number of files > terminal row count)
I'd love to see the hardware (and what that method does to its workload) that supports a find on half a million files every 5-10 seconds. ... if I were your sysadmin and saw you creating this kind of load I'd hunt you down and give you a very stern talking to.
@tink Definitely, if you have a lot of files to inspect, run find many times is not the way to go. My answer helps folks who want to inspect the subdirectories and don't have access to inotify. As I suggested inotify is the best one when you have a lot of files.
16

Use inotifywait from inotify-tools:

sudo apt install inotify-tools

Now create a script myscript.sh that includes hidden files and folders too:

#!/bin/bash while true; do inotifywait -e modify,create,delete,move -r $1 done

Make the script executable with chmod +x myscript.sh

Run it with ./myscript.sh /folder/to/monitor

If you don't provide argument it will use the working directory by default.

Also, you can run several commands adding && \ at the end of the previous command to add the next one:

#!/bin/bash while true; do inotifywait -e modify,create,delete,move -r $1 && \ echo "event" && \ echo "event 2" done

If you don't want to execute any command on events, just run the command directly with the -m modifier so doesn't close:

inotifywait -e modify,create,delete,move -m -r /path/to/your/dir

Improve this answer

1 Comment

Is there any particular advantage of running it in loop instead of using -m flag with pipe eg. inotifywait -e modify -m | while read line; do something_with $line; done as mentioned by @NikolayDimitrov in his comment under other answer ?
9

I have a different suggestion, only for changes in the files, and record history changes

use git

cd /folder_to_monitor git init git add * git commit -m "first snapshot"

so after you make the changes

git diff
Improve this answer

1 Comment

it may be a valid option in some situations. Doesn't deserve a -1
4

Wasn't fanotify supposed to provide that capability eventually? Quoting LWN:

“fanotify has two basic 'modes' directed and global. [...] fanotify global instead indicates that it wants everything on the system and then individually marks inodes that it doesn't care about.”

I lost track what its latest status was, though.

Improve this answer

1 Comment

According to a comment against stackoverflow.com/a/1847268/130352 ... fanotify went into 2.6.36.
3

I was facing the same problem, having a program that was creating some files (starting with a dot) whose content I wanted to manually inspect, but it automatically deleted them again soon after creation.

Using inotify in a loop without the monitor option didn't work for me, because it was too slow and missed events, so I came up with this script:

target="$1" cd "$target" mkdir backup/ inotifywait -e modify,create,delete --monitor -r --include "\..*" "$target" | \ while read line do echo "$line" if [[ "$line" == "$target CREATE "* ]] || [[ "$line" == "$target MODIFY "* ]] then filename=${line#"$target CREATE "} filename=${filename#"$target MODIFY "} cp --verbose "$filename" backup/ fi done
Improve this answer

Comments

3 
inotifywait --format "[%T] %w%f %e" --timefmt "%Y/%m/%d, %H:%M:%S" -m -e modify,create,delete,move -r "/monitored/directory" --outfile /path/to/logdir/file-changes.log

Example output:

... [2024/05/22, 19:28:14] /monitored/directory/somepath/file1.txt MODIFY [2024/05/22, 19:28:15] /monitored/directory/otherpath/file2.txt CREATE [2024/05/22, 19:28:17] /monitored/directory/somepath/file1.txt DELETE ...

Additionally you can use:

  • -d to run in background
  • --no-dereference to know from which symlink a file was changed.
  • --exclude '\.sw.$' to exclude temporary files created by vim (e.g. .swp, .swx e.t.c)
Improve this answer

Comments

2

Especially for large or complex monitoring tasks in which you want to trigger events based on what you see, check out Watchman A file watching service. Here is a simple example to run a tool named minify-css whenever a CSS file is changed:

$ watchman watch ~/src $ watchman -- trigger ~/src buildme '*.css' -- minify-css

It does comprehensive logging, can efficiently handle multiple watches that overlap in a directory structure, can be administered from the command line or via json, and much more. See also

It is available via Debian Sid and Ubuntu 20.04, and has nearly made it in to Fedora twice from what I can see (1450590 and 1564720).

Improve this answer

Comments

0

use watch command, its easy to use and a native Linux command (installed on almost every linux distro)

watch -n <interval> <command> <path>

e.g.

this will show /tmp/test directory changes for every 200 milliseconds

watch -n 0.2 ls -la /tmp/test

for recursive usage you can use watch like below

 watch -n 0.2 find <path>
Improve this answer

3 Comments

This doesn't work. It runs the command if I touch /tmp/test/x but it doesn't if I touch /tmp/test/subdir/y. The question is explicitly looking for a recursive solution.
yes, you're right, we could achieve that functionality with using watch command with find
Now you're just repeating answers from half a decade ago.
-4

I use this to get a quick overview in the current directory:

watch 'find . -printf "%T@ %Tc %p\n" | sort -nr | head '
Improve this answer

1 Comment

Downvoted because 1) it does not answer the question (head would trim the vast majority of the output) and 2) given the number of files and directories of the OP this answer would be impractical even if it were correct because it would periodically look at all the files over again. The OP was looking for a more robust and low-overhead solution.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.