2

I have to do some house cleaning on a folder with millions of files. Ultimately I would like to grab the files and then see if they exist in a database or not. But I need to first grab the files without the program choking.

I tried going the route of using 

var file = Directory.GetFiles(uri, "*.*").FirstOrDefault();

however this takes forever to run. Because of the number of files.

Is there a better way to grab the files that won't take so long?

Improve this question
3
  • 3
    That's exactly why you should split your files by batches in subdirectories. One directory with millions of files is a bad idea. Commented Sep 15, 2016 at 15:15
  • 1
    other than using a direct path to a known file (or narrowing the search filter based on some sort of predictive algorithm), I don't see an easier way. Maybe move it to an async thread so it doesn't lock your program. One suggestion is to load the files into a HashSet<string> if you need to run the method more than once. Then you can run against the hashset for future calls. Commented Sep 15, 2016 at 15:15
  • @LucasTrzesniewski believe me. I sure as heck didn't want this. However I was outvoted. Commented Sep 15, 2016 at 15:17

1 Answer 1

Reset to default
9

Yes, Directory.EnumerateFiles is what you're looking for. This will lazy load your files, so issuing a FirstOrDefault() will not wait for all files to be read.

From the docs:

The EnumerateFiles and GetFiles methods differ as follows: When you use EnumerateFiles, you can start enumerating the collection of names before the whole collection is returned; when you use GetFiles, you must wait for the whole array of names to be returned before you can access the array. Therefore, when you are working with many files and directories, EnumerateFiles can be more efficient.

Aside from this - having a folder with millions of files is just a bad idea.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Believe me. It's a horrible idea. One I didn't want and wanted to have fixed. But I was outvoted on fixing the issue.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.