This week's book giveaway is in the Jython/Python forum.
We're giving away four copies of Murach's Python Programming and have Michael Urban and Joel Murach on-line!
See this thread for details.
Win a copy of Murach's Python Programming this week in the Jython/Python forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Writing huge sized file data(more than 100Mb) to a output stream(converting file data to byte array)  RSS feed

 
Chetan Ram
Greenhorn
Posts: 20
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,

I have a file which is more than 100MB which needs to be written to a output stream (ServletOutputStream) as a response to a request where we can write byte array to a output stream. (outputStream.write(byte[])

Currently, i am using apache commons IOUtils package,

IOUtils.toByteArray(InputStream inputStream)
but this is not working when the file size is more.


Please let me know the better approach with good performance to handle huge files like more than 100MB.

Thanks,
Chinnu
 
Paul Clapham
Sheriff
Posts: 22185
38
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Create a buffer with a reasonable size (maybe 10K bytes or 100K bytes). Write simple code which reads from the input and writes to the output, one buffer at a time. Don't write code which attempts to load the "huge" file completely into memory.
 
Chetan Ram
Greenhorn
Posts: 20
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the reply.

Does this affect performance since every 100 Kb we need to read from input and buffer it and then write to a file every time?
Does this make process slow? Please give me a sample if possible.
 
Paul Clapham
Sheriff
Posts: 22185
38
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I reviewed my post and I don't see where I recommended writing to a file. What I said was
Paul Clapham wrote:Write simple code which reads from the input and writes to the output, one buffer at a time.

In your case "the output" would obviously refer to the ServletOutputStream which you referred to.

As for the "sample", it's hard to give a sample of something which is only about 5 lines of code.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!