Timeline for What's the quickest way to find duplicated files?
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 29, 2021 at 8:40 | comment | added | Ondra Žižka | @TheAmigo, you can't always install tools at will. | |
| Apr 12, 2021 at 15:33 | comment | added | TheAmigo | @OndraŽižka true, and even faster is to first compare file size without even opening the file. If only 2 files have the same size, just compare them (cmp) and if more than 2 have the same size, then do checksums. But why re-invent the wheel, just use one of the other tools that already does that :) | |
| Jan 10, 2021 at 2:58 | comment | added | taltman | This approach was too slow for me. Took >90 minutes to process >380 GB of JPG and MOV files in a nested directory. Used ls -lTR plus the following POSIX awk script to process the same data in 72 seconds: github.com/taltman/scripts/blob/master/unix_utils/… | |
| Jul 23, 2018 at 15:42 | comment | added | heemayl | @Tim I see you've asked a new question already and got the answer :) | |
| Jul 23, 2018 at 2:52 | comment | added | Tim | What does -dD mean to uniq? | |
| Jun 2, 2017 at 0:20 | comment | added | Ondra Žižka | This is not the quickest. For several GB large files, there's no need to hash it whole. You can hash first N kB and then do a full one if same hash is found. | |
| Apr 20, 2016 at 11:37 | history | edited | heemayl | CC BY-SA 3.0 | deleted 16 characters in body |
| Apr 20, 2016 at 5:35 | vote | accept | The One | ||
| Apr 20, 2016 at 4:42 | history | answered | heemayl | CC BY-SA 3.0 |