• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

how to create a new sheet in csv file using IO stream

 
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Everyone,

In the application I am building, I am writing huge data to the csv file. I am using below writer to accomplish my task. In csv files we can write data till 1048576 rows, and later it throws java heap memory issue. Can anyone suggest on how to create a new sheet in the csv, and later move data (after 1048576) to this new sheet.

response.setContentType("text/csv");
response.setHeader("CACHE_CONTROL", " NO_CACHE");
response.setHeader("PRAGMA", " NO_CACHE");
response.setHeader("Content-Disposition",
"attachment; filename= data.csv");
OutputStreamWriter outWriter = new OutputStreamWriter(output);
BufferedWriter writer = new BufferedWriter(outWriter, (int) MEG);
 
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
CSVs have no concept of "sheets" like spreadsheets do. While I think a CSV might not be the best choice of file format for so much data, have you increased the servlet container's memory allocation?
 
shashank beerla
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am not suppose to increase the container memory allocation as the production server has the same I have currently in my local. What will be the best format? to my knowledge you write more dta to csv than excel or any other format
 
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Much depends on the control you have on the client configuration and you don't say which client you are using to issue the HTTP request. Setting the client to use 'chunked' mode allows the server to send a chunk of data at a time so limiting the memory footprint. As a starting point look at the Javadoc for class HttpURLConnection; this should give you terms to look for using Google.
 
shashank beerla
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hey richard
Thanks for your reply. But in my case I have to create new sheet in the csv file using a response object ? can we or not?
 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

shashank beerla wrote:hey richard
Thanks for your reply. But in my case I have to create new sheet in the csv file using a response object ? can we or not?



As has already been said, CSV has no concept of 'sheets' and even if it had I don't see how this would reduce the memory footprint.
 
Rancher
Posts: 1044
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator


What size is it? Do you really need it that large?
 
Put a gun against his head, pulled my trigger, now he's dead, that tiny ad sure bled
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic