• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

OutOfMemoryError: Java heap space error when exporting more than 65MB file

 
monica singh
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,

My requirement is to export 65Mb file in a CSV format.There is no possibility for me to increase the heap size.
But ,I get the following error which is below the code while doing an export.
Whole 65mb ,I am putting into the response stream.I am not sure whether this is allowed or not.If not,please tell me the alternatives.

ServletOutputStream out = null;
final FacesContext faces = FacesContext.getCurrentInstance();
final HttpServletResponse response = (HttpServletResponse) faces
.getExternalContext().getResponse();
response.setContentType( "application/download" );
response.setHeader( ...................);
final byte[] byteArray = eachLine.toString().getBytes();
response.setContentLength( byteArray.length );
out.write(byteArray);

java.lang.OutOfMemoryError: Java heap space
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:95)
at org.apache.myfaces.webapp.filter.ExtensionsResponseWrapper$MyServletOutputStream.write(ExtensionsResponseWrapper.java:138)
at org.ajax4jsf.io.FastBufferOutputStream.writeTo(FastBufferOutputStream.java:151)
at org.ajax4jsf.framework.ajax.xmlfilter.FilterServletResponseWrapper.sendContent(FilterServletResponseWrapper.java:469)
at org.ajax4jsf.framework.ajax.xmlfilter.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:193)
at org.ajax4jsf.framework.ajax.xmlfilter.BaseFilter.doFilter(BaseFilter.java:220)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:100)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:147)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 
David O'Meara
Rancher
Posts: 13459
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It looks like byteArray contains the entire response and that you are trying to send the entire CSV response at once.
You may want to try wrapping out in a BufferedWriter (although it should already be, but a bit hard tell in your code)
and then write the response to the output stream in parts rather than all at once.
 
monica singh
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
How would I wrap a bufferedwriter here.Please help.thanks
 
David O'Meara
Rancher
Posts: 13459
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You haven't shown the line of code where you assign a value to out, but typically it is easy as new BufferedWriter(out);
 
monica singh
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Here is how m initializing out - out = response.getOutputStream();
I shal check now with buffered writer and see whether the problem exists
 
monica singh
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I could see that new BufferedWritter constructor doesnot take out here which is a servletoutputstream.Above thing didnot work
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13074
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
In the following code:


what is eachLine, and why are you creating an entirely new object from it with toString() when all you want is the bytes?

Does this CSV data exist in a file somewhere or is some other process creating it in memory?

Does your program create the CSV data? if not, what is the source?

Bill
 
monica singh
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There is a file in some format in some location.
eachLine is the every line that I read from the buffered reader of that file.
I read the file line by line which is a string buffer which is converted to string and write it to csv.
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13074
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I read the file line by line which is a string buffer which is converted to string and write it to csv.


Why do you find it necessary to set the content length? Without that requirement, you could write to the response stream as each line is read.

It is still not clear - does the file exist in some other format that you have to convert line by line?

Bill
 
David O'Meara
Rancher
Posts: 13459
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
or read and return the source file length (again, only if you are not providing any conversion on the data)
 
Bauke Scholtz
Ranch Hand
Posts: 2458
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic