1196

I want to traverse all subdirectories, except the node_modules directory.

Improve this question
3
  • 3
    See superuser.com/q/66715/59933 Commented Jul 3, 2011 at 20:55
  • 94
    If you are grepping for code in a git repository and node_modules is in your .gitignore, git grep "STUFF" is the easiest way. git grep searches the tracked files in the working tree, ignoring everything from .gitignore Commented Dec 22, 2016 at 21:51
  • 10
    An example for node: grep -R --exclude-dir={node_modules,bower_components} "MyString" | cut -c1-"$COLUMNS" -- further you could always alias this in the shell to 'nodegrep' or whatever and use a command argument as string input.. Commented Mar 21, 2018 at 21:59

15 Answers 15

Reset to default
1465

Recent versions of GNU Grep (>= 2.5.2) provide:

--exclude-dir=dir

which excludes directories matching the pattern dir from recursive directory searches.

So you can do:

grep -R --exclude-dir=node_modules 'some pattern' /path/to/search

For a bit more information regarding syntax and usage see

For older GNU Greps and POSIX Grep, use find as suggested in other answers.

Or just use ack (Edit: or The Silver Searcher) and be done with it!

Improve this answer
Sign up to request clarification or add additional context in comments.

20 Comments

Syntax for the impatient: --exclude-dir=dir uses grep's regular expression patterns, not shell's file globbing. Patterns work on paths relative to your current directory. So use pattern --exclude-dir=dir, not --exclude-dir="/root/dir/*".
If you wish to exclude multiple dirs from the search, is there a better option than to use : $ grep -r --exclude-dir=dir1 --exclude-dir=dir2 "string" /path/to/search/dir ?
I probably spent way too much time on this than any sane person, but I can't for the life of me figure out how to exclude a subdirectory from the search - grep -r --exclude-dir=public keyword . works, but grep -r --exclude-dir='public/dist' keyword . does not. I tried adding regex wildcards, escaping characters etc, but nothing seems to help.
Exclude multiple directories like so: grep -r "Request" . --exclude-dir={node_modules,git,build}
@dkobozev to exclude specific subdirectories you must start with the dot, so using grep -r --exclude-dir='./public/dist' keyword should be what you're looking for
|
631

SOLUTION 1 (combine find and grep)

The purpose of this solution is not to deal with grep performance but to show a portable solution : should also work with busybox or GNU version older than 2.5.

Use find, for excluding directories foo and bar :

find /dir \( -name foo -prune \) -o \( -name bar -prune \) -o -name "*.sh" -print

Then combine find and the non-recursive use of grep, as a portable solution :

find /dir \( -name node_modules -prune \) -o -name "*.sh" -exec grep --color -Hn "your text to find" {} 2>/dev/null \;

SOLUTION 2 (recursive use of grep, using the -R and --exclude-dir option of grep):

You know this solution already (recursive search), but I add it since it's the most recent and efficient solution. Note this is a less portable solution but more human-readable.

grep -R --exclude-dir=node_modules 'some pattern' /path/to/search

To exclude multiple directories, use --exclude-dir as:

--exclude-dir={node_modules,dir1,dir2,dir3}

Note the "-R" option for a deference recursive file reading, i.e. following ALL sym links, unlike "-r".

SOLUTION 3 (Ag)

If you frequently search through code, Ag (The Silver Searcher) is a much faster alternative to grep, that's customized for searching code. For instance, it automatically ignores files and directories listed in .gitignore, so you don't have to keep passing the same cumbersome exclude options to grep or find.

Improve this answer

7 Comments

this combination searches faster than --exclude-dir=dir and it shows results with colors - easy to read
"this combination" find ... -exec is not faster than grep --exclude-dir for me. Huge advantage to grep (about five times faster with 26k+ files, filtered out of 38k+ on an HDD), unless you replace the \; with + for the find/exec combo. Then grep is "only" about 30% faster. The grep syntax is also human-readble :).
Agreed, since this is obvious. Some busyboxes does not have the GREP command.
grep --exclude-dir=dir -rin "text" . works fine for me. ag text . (silver_searcher) works superb
also noting that you can exclude multiple with --exclude-dir={dir1,dir2}
|
133

This syntax

--exclude-dir={dir1,dir2}

is expanded by the shell (e.g. Bash), not by grep, into this:

--exclude-dir=dir1 --exclude-dir=dir2

Quoting will prevent the shell from expanding it, so this won't work:

--exclude-dir='{dir1,dir2}' <-- this won't work

The patterns used with --exclude-dir are the same kind of patterns described in the man page for the --exclude option:

--exclude=GLOB Skip files whose base name matches GLOB (using wildcard matching). A file-name glob can use *, ?, and [...] as wildcards, and \ to quote a wildcard or backslash character literally.

The shell will generally try to expand such a pattern itself, so to avoid this, you should quote it:

--exclude-dir='dir?'

You can use the curly braces and quoted exclude patterns together like this:

--exclude-dir={'dir?','dir??'}
Improve this answer

Comments

127

If you want to exclude multiple directories:

"r" for recursive, "l" to print only names of files containing matches and "i" to ignore case distinctions :

grep -rli --exclude-dir={dir1,dir2,dir3} keyword /path/to/search

Example : I want to find files that contain the word 'hello'. I want to search in all my linux directories except proc directory, boot directory, sys directory and root directory :

grep -rli --exclude-dir={proc,boot,root,sys} hello /

Note : The example above needs to be root

Note 2 (according to @skplunkerin) : do not add spaces after the commas in {dir1,dir2,dir3}

Improve this answer

3 Comments

NOTE: do not add spaces after the commas in {dir1,dir2,dir3}
Thanks, handy when grep'ing through SVN workspace: grep -Irsn --exclude-dir=.svn 'foo' .
You can just provide the --exclude-dir option multiple times.
32

If you are grepping for code in a git repository and node_modules is in your .gitignore, you can use git grep. git grep searches the tracked files in the working tree, ignoring everything from .gitignore

git grep "STUFF"
Improve this answer

1 Comment

This is an incredible way to limit wading through tons of generated files (dist, bin, etc.) and it pipes into a utility like less automagically so you have a scrollable search result. Fantastic!
27

Many correct answers have been given here, but I'm adding this one to emphasize one point which caused some rushed attempts to fail before: exclude-dir takes a pattern, not a path to a directory.

Say your search is:

grep -r myobject

And you notice that your output is cluttered with results from the src/other/objects-folder. This command will not give you the intended result:

grep -r myobject --exclude-dir=src/other/objects-folder

And you may wonder why exclude-dir isn't working! To actually exclude results from the objects-folder, simply do this:

grep -r myobject --exclude-dir=objects-folder

In other words, just use the folder name, not the path. Obvious once you know it.

From the man page:

--exclude-dir=GLOB
Skip any command-line directory with a name suffix that matches the pattern GLOB. When searching recursively, skip any subdirectory whose base name matches GLOB. Ignore any redundant trailing slashes in GLOB.

Improve this answer

3 Comments

Why on the planet didn't I scroll down to this answer before I posted my comment/question up above? I unfortunately have the bad habit to ignore answers with less upvotes, but this explains what I was doing wrong, so thanks Nagev.
I would go crazy because I used the full path to a subfolder, but it did not work. After I saw this comment, I used the folder name ONLY and it finally worked! THANKS
grep --exclude-dir taking dirnames rather than paths is really insane. e.g. consider you have the same dirname multiple times in different subtrees but you want to exclude only one of them. Not to mention how dirty it is to specify just the name of some distant subdirectory - a full path would be much clearer.
17

Frequently use this:

grep can be used in conjunction with -r (recursive), i (ignore case) and -o (prints only matching part of lines). To exclude files use --exclude and to exclude directories use --exclude-dir.

Putting it together you end up with something like:

grep -rio --exclude={filenames comma separated} \ --exclude-dir={directory names comma separated} <search term> <location>

Describing it makes it sound far more complicated than it actually is. Easier to illustrate with a simple example.

Example:

Suppose I am searching for current project for all places where I explicitly set the string value debugger during a debugging session, and now wish to review / remove.

I write a script called findDebugger.sh and use grep to find all occurrences. However:

For file exclusions - I wish to ensure that .eslintrc is ignored (this actually has a linting rule about debugger so should be excluded). Likewise, I don't want my own script to be referenced in any results.

For directory exclusions - I wish to exclude node_modules as it contains lots of libraries that do reference debugger and I am not interested in those results. Also I just wish to omit .idea and .git hidden directories because I don't care about those search locations either, and wish to keep the search performant.

So here is the result - I create a script called findDebugger.sh with:

#!/usr/bin/env bash grep -rio --exclude={.eslintrc,findDebugger.sh} \ --exclude-dir={node_modules,.idea,.git} debugger .
Improve this answer

4 Comments

I believe the "r" option should be printed with an upper case "-R".
Interesting. "r" has always worked for me on nix and mac.
When I wrote my answer, I used -R (I don't recall why now). I typically use -r. It turns out that the uppercase version follows symlinks. TIL.
@Johnsyweb - thanks. upvoted your answer - don't recall when, likely in 2016 when I added this one :)
4

You could try something like grep -R search . | grep -v '^node_modules/.*'

Improve this answer

4 Comments

Not such a good solution in some cases. For example: If 'node_modules' directory is a huge one with lots of false positive matches (hence the need to filter out the directory) then the first grep is wasting a lot of time searching through a sub-directory and THEN the second grep filtering out the matches. It's faster to exclude node_modules in the first grep itself.
i don't care about the slowness, I can look at the command and know what it does
Ditto for Guru's comment. A grep of /var hangs when it hits /var/run in my case. Hence the reason I want to avoid the directory in the first place.
--exclude-dir is the best solution as of 2016.
4

Very useful, especially for those dealing with Node.js where we want to avoid searching inside "node_modules":

find ./ -not -path "*/node_modules/*" -name "*.js" | xargs grep keyword
Improve this answer

Comments

3

Step 1:

vim ~/.bash_profile

search() { grep -InH -r --exclude-dir=*build* -e "$1" . }

Step 2:

source ~/.bash_profile

Usage:

search "<string_to_be_searched>"

Improve this answer

Comments

2

A simple working command:

root/dspace# grep -r --exclude-dir={log,assetstore} "creativecommons.org"

Above I grep for text "creativecommons.org" in current directory "dspace" and exclude dirs {log,assetstore}.

Done.

Improve this answer

2 Comments

Neat, including several directories in brackets
This was already mentioned in three existing answers. Suggest deletion to clean up this thread and remove noise/redundancy.
1

Try this:

find /path/to/search -type f -not -path '*/folder-to-exclude/*' -exec grep "search-pattern" {} +

or multiple folders:

find /home/user/projects -type f -not -path '*/node_modules/*' -not -path '*/logs/*' -exec grep "charmander" {} +
Improve this answer

Comments

0 
find . ! -name "node_modules" -type d
Improve this answer

Comments

-2

This one works for me:

grep <stuff> -R --exclude-dir=<your_dir>
Improve this answer

1 Comment

How is this answer different from what has already been posted?
-7

A simpler way would be to filter your results using "grep -v".

grep -i needle -R * | grep -v node_modules

Improve this answer

1 Comment

This is effectively the same answer DipSwitch provided 3 years earlier. It has the same problems, too.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.