20

When I am trying to download a large file which is of 260MB from server, I get this error: java.lang.OutOfMemoryError: Java heap space. I am sure my heap size is less than 252MB. Is there any way I can download large files without increasing heap size?

How I can download large files without getting this issue? My code is given below:

String path= "C:/temp.zip"; response.addHeader("Content-Disposition", "attachment; filename=\"test.zip\""); byte[] buf = new byte[1024]; try { File file = new File(path); long length = file.length(); BufferedInputStream in = new BufferedInputStream(new FileInputStream(file)); ServletOutputStream out = response.getOutputStream(); while ((in != null) && ((length = in.read(buf)) != -1)) { out.write(buf, 0, (int) length); } in.close(); out.close();
Improve this question
2
  • Are you able to increase the heap size on the JVM? ie: java -Xm512m -Xmx512m myClass Commented Aug 18, 2011 at 11:41
  • yes, but i would like to know if i can do this without increasing jvm heap size, as we have a requirement to download large files of about 1gb to 10 gb Commented Aug 18, 2011 at 11:50

4 Answers 4

Reset to default
30

There are 2 places where I can see you could potentially be building up memory usage:

  1. In the buffer reading your input file.
  2. In the buffer writing to your output stream (HTTPOutputStream?)

For #1 I would suggest reading directly from the file via FileInputStream without the BufferedInputStream. Try this first and see if it resolves your issue. ie:

FileInputStream in = new FileInputStream(file);

instead of:

BufferedInputStream in = new BufferedInputStream(new FileInputStream(file));

If #1 does not resolve the issue, you could try periodically flushing the output stream after so much data is written (decrease chunk size if necessary):

ie:

try { FileInputStream fileInputStream = new FileInputStream(file); byte[] buf=new byte[8192]; int bytesread = 0, bytesBuffered = 0; while( (bytesread = fileInputStream.read( buf )) > -1 ) { out.write( buf, 0, bytesread ); bytesBuffered += bytesread; if (bytesBuffered > 1024 * 1024) { //flush after 1MB bytesBuffered = 0; out.flush(); } } } finally { if (out != null) { out.flush(); } }
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

One small issue is there with the code above. We need to add one more flush in the finally block if a try-catch which surrounds the code or at the end of the while loop. Else it will not flush the complete bytes. Else may cause some issues some times.
Wowww ... Now its perfect :)
Is there any way to download 1g file?
6

Unfortunately you have not mentioned what type out is. If you have memory issues I guess it is ByteArrayOutpoutStream. So, replace it by FileOutputStream and write the byte you are downloading directly to file.

BTW, do not use read() method that reads byte-by-byte. Use read(byte[] arr) instead. This is much faster.

Improve this answer

3 Comments

Judging by the "response.setContentType" and "response.setHeader" calls, this is a servlet - ie. he's streaming a file to a web client and "out" is the HttpOutputStream.
thank you alex, but my main issue is with downloading larger files which can be 1gb, 10 gb size files. Here i get out of memory exception. How can i handle this without increasing the heap size....
OK, now you have updated you code and the question is much more clear. I thought that your program is downloading file from remote URL but you are asking opposite question: your servlet provides ability to download file. So, I think that your problem is that you are not calling out.flush() after each out.write(), so all bytes you are reading are stored in memory and are not sent to network until you are closing the output stream. Try to flush and see what's happening.
0

First you can remove the (in != null) from your while statement, it's unnecessary. Second, try removing the BufferedInputStream and just do:

FileInputStream in = new FileInputStream(file);
Improve this answer

Comments

0

There's nothing wrong (in regard to memory usage) with the code you're show. Either the servlet container is configured to buffer the entire response (look at the web.xml configuration), or the memory is being leaked elsewhere.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.