File.ReadAllLines or Stream Reader

Question

We can read file either by using StreamReader or by using File.ReadAllLines.

For example I want to load each line into a List or string[] for further manipulation on each line.

string[] lines = File.ReadAllLines(@"C:\\file.txt"); foreach(string line in lines) { //DoSomething(line); }

or

using (StreamReader reader = new StreamReader("file.txt")) { string line; while ((line = reader.ReadLine()) != null) { //DoSomething(line); or //save line into List<string> } } //if list is created loop through list here

Application come across different size of text file. Which could grow from few KBs to MBs occasionally.

My question is that which one is preferred way and why one should be preferred over other?

@Steve, speed and memory both. If larger size text file comes then application should efficiently read all lines. — Hassan Nisar
– Hassan Nisar, Commented Jun 2, 2014 at 7:59

Matthew Watson · Accepted Answer · 2014-06-02 08:11:16Z

43

If you want to process each line of a text file without loading the entire file into memory, the best approach is like this:

foreach (var line in File.ReadLines("Filename")) { // ...process line. }

This avoids loading the entire file, and uses an existing .Net function to do so.

However, if for some reason you need to store all the strings in an array, you're best off just using File.ReadAllLines() - but if you are only using foreach to access the data in the array, then use File.ReadLines().

edited Jun 2, 2014 at 8:11

answered Jun 2, 2014 at 8:05

Matthew Watson

111k12 gold badges179 silver badges301 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Matthew Watson Over a year ago

@downvoter: Care to explain? Seems odd to downvote the correct answer... ;)

Nicholas Carey Over a year ago

Probably downvoted because File.ReadLines() slurp the entire file into memory (a string[]).

Matthew Watson Over a year ago

@NicholasCarey No, it's the version that DOESN'T do that: public static IEnumerable<string> ReadLines()... hence my comment "without loading the entire file into memory"

TripleEEE Over a year ago

@MatthewWatson I agree with you, though it's an old topic - upvoted for anybody going to read this and is beeing confused.

tronman Jul 17 at 15:37

Here's a good analysis of what File.ReadLines() is doing and why it is memory efficient: medium.com/@nuno.caneco/…

Sam Axe · Accepted Answer · 2014-06-02 08:03:32Z

Microsoft uses a StreamReader in File.ReadAllLines:

 private static String[] InternalReadAllLines(String path, Encoding encoding) { Contract.Requires(path != null); Contract.Requires(encoding != null); Contract.Requires(path.Length != 0); String line; List<String> lines = new List<String>(); using (StreamReader sr = new StreamReader(path, encoding)) while ((line = sr.ReadLine()) != null) lines.Add(line); return lines.ToArray(); }

Yousha Aleayoub · Accepted Answer · 2021-07-26 12:06:08Z

The StreamReader read the file line by line, it will consume less memory. Whereas, File.ReadAllLines read all lines at once and store it into string[], it will consume more memory. And if that string[] is larger than int.maxvalue then that will produce memory overflow(limit of 32bit OS).

So, for bigger files StreamReader will be more efficient.

Given Sam's answer about ReadAllLines using StreamReader internally, is this answer invalid?
This answer is correct. But you can also use File.ReadLines(), which uses a StreamReader under the hood and maintains memory efficiency.

Collectives™ on Stack Overflow

File.ReadAllLines or Stream Reader

3 Answers 3

5 Comments

Comments

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

Comments

2 Comments

Linked

Related