This week's book giveaway is in the Jython/Python forum.
We're giving away four copies of Murach's Python Programming and have Michael Urban and Joel Murach on-line!
See this thread for details.
Win a copy of Murach's Python Programming this week in the Jython/Python forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

how to create a new sheet in csv file using IO stream  RSS feed

 
shashank beerla
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Everyone,

In the application I am building, I am writing huge data to the csv file. I am using below writer to accomplish my task. In csv files we can write data till 1048576 rows, and later it throws java heap memory issue. Can anyone suggest on how to create a new sheet in the csv, and later move data (after 1048576) to this new sheet.

response.setContentType("text/csv");
response.setHeader("CACHE_CONTROL", " NO_CACHE");
response.setHeader("PRAGMA", " NO_CACHE");
response.setHeader("Content-Disposition",
"attachment; filename= data.csv");
OutputStreamWriter outWriter = new OutputStreamWriter(output);
BufferedWriter writer = new BufferedWriter(outWriter, (int) MEG);
 
Ulf Dittmer
Rancher
Posts: 42970
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
CSVs have no concept of "sheets" like spreadsheets do. While I think a CSV might not be the best choice of file format for so much data, have you increased the servlet container's memory allocation?
 
shashank beerla
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am not suppose to increase the container memory allocation as the production server has the same I have currently in my local. What will be the best format? to my knowledge you write more dta to csv than excel or any other format
 
Richard Tookey
Bartender
Posts: 1166
17
Java Linux Netbeans IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Much depends on the control you have on the client configuration and you don't say which client you are using to issue the HTTP request. Setting the client to use 'chunked' mode allows the server to send a chunk of data at a time so limiting the memory footprint. As a starting point look at the Javadoc for class HttpURLConnection; this should give you terms to look for using Google.
 
shashank beerla
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hey richard
Thanks for your reply. But in my case I have to create new sheet in the csv file using a response object ? can we or not?
 
Richard Tookey
Bartender
Posts: 1166
17
Java Linux Netbeans IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
shashank beerla wrote:hey richard
Thanks for your reply. But in my case I have to create new sheet in the csv file using a response object ? can we or not?


As has already been said, CSV has no concept of 'sheets' and even if it had I don't see how this would reduce the memory footprint.
 
Ivan Jozsef Balazs
Rancher
Posts: 992
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator


What size is it? Do you really need it that large?
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!