1

I want to read a huge .txt file and I'm getting a memory overflow because of its sheer size.

Any help?

 private void button1_Click(object sender, EventArgs e) { using (var Reader = new StreamReader(@"C:\Test.txt")) { textBox1.Text += Reader.ReadLine(); } }

Text file is just:

Line1 Line2 Line3

Literally like that.

I want to load the text file to a multiline textbox just as it is, 100% copy.

Improve this question
6
  • 2
    Read and process it one line at a time, or break it into chunks and deal with the chunks individually. You can also show us the code you have, and tell us what you are trying to accomplish with it. Commented May 19, 2010 at 18:10
  • How would I do that? I saw a similar question here in SO but I can't find it. Search doesn't work well for me. Commented May 19, 2010 at 18:11
  • @Robert If you post this, you'll get an upvote :) Commented May 19, 2010 at 18:12
  • @Sergio search tip : try keyword1 +keyword2 etc : ) Commented May 19, 2010 at 18:12
  • How are you getting a memory overflow when you're reading only one line of text? Commented May 19, 2010 at 18:28

4 Answers 4

Reset to default
3

Firstly, the code you posted will only put the first line of the file into the TextBox. What you want is this:

using (var reader = new StreamReader(@"C:\Test.txt")) { while (!reader.EndOfStream) textBox1.Text += reader.ReadLine(); }

Now as for the OutOfMemoryException: I haven't tested this, but have you tried the TextBox.AppendText method instead of using +=? The latter will certainly be allocating a ton of strings, most of which are going to be nearly the length of the entire file by the time you near the end of the file.

For all I know, AppendText does this as well; but its existence leads me to suspect it's put there to deal with this scenario. I could be wrong -- like I said, haven't tested personally.

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

So the reader needs a call to EndOfStream to consume the previous line, so that the next line can be read? I didn't know that. Seems a bit counterintuitive.
@Robert Harvey: No, it doesn't need to call EndOfStream to consume the previous line; it just needs to check that property to know when to stop. The issue was that the OP was not utilizing a loop at all; he was continuously creating a new StreamReader and calling ReadLine once per instance.
The += on the text section of the text box, is just a += on a string. That is the worst way to append text to a string in .net. As the size of the text increases the memory usage gets worse and worse.
@Justin: I realize that; this is why I suggested trying AppendText instead. But I had to be honest and admit that I'm not really sure AppendText isn't just calling += itself.
Tested this with a file of size 7 GB on x64 machine. worked good,
3

You'll get much faster performance with the following:

textBox1.Text = File.ReadAllText(@"C:\Test.txt");

It might also help with your memory problem, since you're wasting an enormous amount of memory by allocating successively larger strings with each line read.

Granted, the GC should be collecting the older strings before you see an OutOfMemoryException, but I'd give the above a shot anyway.

Improve this answer

Comments

2

First use a rich text box instead of a regular text box. They're much better equiped for the large amounts of data you're using. However you still need to read the data in. 

// use a string builer, the += on that many strings increasing in size // is causing massive memory hoggage and very well could be part of your problem StringBuilder sb = new StringBuilder(); // open a stream reader using (var reader = new StreamReader(@"C:\Test.txt")) { // read through the stream loading up the string builder while (!reader.EndOfStream) { sb.Append( reader.ReadLine() ); } } // set the text and null the string builder for GC textBox1.Text = sb.ToString(); sb = null;
Improve this answer

Comments

2

Read and process it one line at a time, or break it into chunks and deal with the chunks individually. You can also show us the code you have, and tell us what you are trying to accomplish with it.

Here is an example: C# Read Text File Containing Data Delimited By Tabs Notice the ReadLine() and WriteLine() statements.

TextBox is severely limited by the number of characters it can hold. You can try using the AppendText() method on a RichTextBox instead.

Improve this answer

4 Comments

Robert, it's reading the first line "Line1" correctly, but it doesn't grab Line2 or Line3. I'm trying to create a literal copy of the text file, formatting and all.
@Sergio: Try the RichTextBox. If the file isn't amazingly large, it should work; RichTextBox is far better suited for this sort of thing.
The ReadLine() method is only returning my first line over and over and over again; how can I tell it to read the first line then continue moving along?
@Sergio: You need to do this in a while loop, checking the EndOfStream property on each iteration. Take a look at my answer.