2

Trying to search for all files that are excluded from list.txt using the following from the command line

find . -type f -name "*.ext" $(printf "! -name %s " $(cat list.txt)) 

I get the following result

-bash: /usr/bin/find: Argument list too long 

I have also tried with xargs but not sure if I am using it correctly. Any help would be appreciated.

5
  • 1
    Can you show us what you tried with xargs? Commented Feb 25, 2015 at 19:37
  • Obvious question would be how big your list is - if you chop off just the first 10 lines does it work then, or is there something wrong with the idea in concept as well as quantity? Also, you might try preceding the command line with echo to sort of produce a "what it might do" output. Commented Feb 25, 2015 at 19:37
  • The list has over 1M lines Commented Feb 25, 2015 at 19:40
  • 1
    I tried just piping to xargs by adding -print0 | xargs -0 Commented Feb 25, 2015 at 19:41
  • replace ".ext" with '.ext' ? Commented Jul 15, 2015 at 13:51

3 Answers 3

2

I would pipe to grep:

find -type f -name '*.ext' | grep -vFf list.txt 

When passing -f grep reads search patterns from a file. -v negates the search. Usually grep treats search patterns as regular expressions. If you -F grep will treat those as fixed strings.

Sign up to request clarification or add additional context in comments.

3 Comments

This makes sense, but I got the following output ------------------------- "grep: Unmatched [ or [^"
What if you try find ... | grep -vFf list.txt ?
Yes! This works! The -vFf helped to deal with the whitespace I assume. Thank you!
0

Can you try this?

find . -type f -name "*.ext" -exec grep -v "^{}$" list.txt && echo {} \;

1 Comment

got the following : find: missing argument to `-exec'
0

It's not completely clear what you're aiming to do with your command, so these remarks are a little generic.

If you want to grep a large number of files for the word foo, for example, then you'd do something like

find . -type f -name \*.ext | xargs grep foo 

The important thing here is that the \* is passed to find for it to expand, and it is not expanded by the shell. The 'Argument list too long' error is probably the result of the shell expanding your "*.ext" and trying to pass that long list as a bunch of arguments to find. This may be the key thing you're missing in your experiments here.

The way that xargs works is that it reads arguments from the stdin (in this case the list of files produced by find), and calls the given function repeatedly with as many of these arguments as will fit. If the strings being sent to xargs are at all unusual, you should look at the -0 option for xargs.

Another way to use find, which is more natural in some circumstances, is

find . -type f <whatever> | while read f; do some-command $f; done 

Here, while again reads lines one at a time, assigning them in turn to f, which are processed one at a time in the loop.

2 Comments

How would this exclude the files from list.txt?
@biggthed Ahhh, I see what you're doing with list.txt! (doh: the bit I was focusing on was the argument-number overflow that might result from "*.ext") The way I'd do that is not by constructing a mega argument list for find, but by find . -name \*.ext | cat - list.txt | sort | uniq -u | .... The cat - list.txt results in a list where all the files in list.txt are duplicated, which are duly removed by uniq -u

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.