To get X number of files in a directory, I can do:
$ ls -U | head -40000 How would I then delete these 40,000 files? For example, something like:
$ "rm -rf" (ls -U | head -40000) 
4 Answers
The tool you need for this is xargs. It will convert standard input into arguments to a command that you specify. Each line of the input is treated as a single argument.
Thus, something like this would work (see the comment below, though, ls shouldn't be parsed this way normally):
ls -U | head -40000 | xargs rm -rf I would recommend before trying this to start with a small head size and use xargs echo to print out the filenames being passed so you understand what you'll be deleting.
Be aware if you have files with weird characters that this can sometimes be a problem. If you are on a modern GNU system you may also wish to use the arguments to these commands that use null characters to separate each element. Since a filename cannot contain a null character that will safely parse all possible names. I am not aware of a simple way to take the top X items when they are zero separated.
So, for example you can use this to delete all files in a directory
find . -maxdepth 1 -print0 | xargs -0 rm -rf 
2 Comments
kojiro
Parsing
ls is an antipattern and should be avoided in any automation. (I realize this was in the question, but readers are likely to skip directly to the answer.)
Adam B
True. Updated the answer to have that warning before I use
lsUse a bash array and slice it. If the number and size of arguments is likely to get close to the system's limits, you can still use xargs to split up the remainder.
files=( * ) printf '%s\0' "${files[@]:0:40000}" | xargs -0 rm 
10 Comments
hek2mgl
Have you tested that with 40.000 filenames having arbitrary length? Do you think
rm can consume 40.000 arguments?kojiro
@hek2mgl no. How shall I produce names with arbitrary length? More seriously, I could test it, and it could work here because I have a particular implementation of
exec, and it could still fail for OP, who may have a different one. On OS X, ARG_MAX is ~264k.hek2mgl
No, this (a) won't work and (b) even if it would work on a future system it never would scale. I don't wan't to be the smart ass but that's simply the truth. Btw I should have simply used "maximum" instead of "arbitrary" length above.
kojiro
@hek2mgl in what sense would it not scale?
hek2mgl
I mean shell variables have a limited capacity. Also the maximum length of a command line is limited. Even if upcoming operating systems increase this limit it will be limited. Conclusion: You can't store that in a bash variable, you can't call rm just once. (I mean reliably). You need to call it as many times as needed to process all arguments, always passing the possible number of arguments allowed to not hit the character limit. Actually this is was
xargs is doing ;) |
What about using awk as the filter?
find "$FOLDER" -maxdepth 1 -mindepth 1 -print0 \ | awk -v limit=40000 'NR<=limit;NR>limit{exit}' RS="\0" ORS="\0" \ | xargs -0 rm -rf It will reliably remove at most 40.000 files (or folders). Reliably means regardless of which characters the filenames may contain.
Btw, to get the number of files in a directory reliably you can do:
find FOLDER -mindepth 1 -maxdepth 1 -printf '.' | wc -c Comments
I ended up doing this since my folders were named with sequential numbers. This should also work for alphabetical folders:
ls -r releases/ | sed '1,3d' | xargs -I {} rm -rf releases/{} Details:
- list all the items in the
releases/folder in reverse order - slice off the first 3 items (which would be the newest if numeric/alpha naming)
- for each item,
rmit
In your case, you can replace ls -r with ls -U and 1,3d with 1,40000d. That should be the same, I believe.
1 Comment
Badr B
This works, but instead of deleting 3 files, it left only 3 files.
-Uoption tolsin this example will return directory contents unsorted (as they are "natively" stored). It will also not show dot files. You may want another option depending on your needs.