What's the most efficient way to write large text file in java?

Question

I'm trying to write a file with some amount of data using this:

public static <T extends SomeClass> void writeFile(String buffer, Class<T> clazz, int fileNumber) { String fileType = ".txt"; File file = new File(clazz.getName()+fileNumber+fileType); PrintWriter printWriter = null; try { FileWriter writer = new FileWriter(file); printWriter = new PrintWriter(writer); printWriter.print(buffer);//error occurs here printWriter.flush(); printWriter.close(); System.out.println("created file: "+file.getName()); } catch (IOException e) { e.printStackTrace(); } finally{ if(printWriter!=null){ printWriter.flush(); printWriter.close(); } } System.out.println("Done!"); }

The buffer string contains +-6mb of data, and when i run the code i get a java.lang.OutOfMemoryError exactly in buffer.

Have you given your Java VM a reasonable amount of memory with the -Xmx command line option? — Simon Nickerson
– Simon Nickerson, Commented Oct 13, 2011 at 11:31
Did you try writing the buffer in smaller blocks instead of a single call? — millimoose
– millimoose, Commented Oct 13, 2011 at 11:33
Incidentally, i believe the reason that PrintWriter.write uses memory is that it encodes the whole string into a byte array, and then writes that. — Tom Anderson
– Tom Anderson, Commented Oct 13, 2011 at 13:06

Sibbo · Accepted Answer · 2011-10-13 11:35:50Z

1

What about replacing printWriter.print(buffer); with:

for (int i = 0; i < buffer.length; i += 100) { int end = i + 100; if (end >= buffer.length) { end = buffer.length; } printWriter.print(buffer.substring(i, end); printWriter.flush(); }

answered Oct 13, 2011 at 11:35

Sibbo

3,9323 gold badges25 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Thom Over a year ago

Programmer's these days are used to having so much memory, they never think about conserving it. Life is about balance.

Gianpaolo Di Nino Over a year ago

@Thom: agree with you, but balance means using as max as resource you have in the right way. ;)

Thom Over a year ago

Balance means using the right amount of resources the right way.

Gianpaolo Di Nino · Accepted Answer · 2011-10-13 11:33:15Z

Since 6mb is not so much "data" I think you should increase your java VM memory,

take a look here

http://confluence.atlassian.com/display/DOC/Fix+Out+of+Memory+Errors+by+Increasing+Available+Memory

Collectives™ on Stack Overflow

What's the most efficient way to write large text file in java?

2 Answers 2

3 Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Linked

Related