• Post Reply Bookmark Topic Watch Topic
  • New Topic

Heap size increases when writing XML to the File  RSS feed

 
batuk chatuk
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am converting an object (ArrayList) in XML and then storing that XML as a file on local machine. When there are large number of data, the XML takes too long to write in a File or doesnt write at all (gives some exception).

xmlSource = xmlConverter.convert(object); // object here is no.of ArrayLists..

FileWriter fw = new FileWriter(f);
fw.write(xmlConverter.getXML());

fw.write() takes too much time. Is there any other way which doesnt eat up so much memory ???
 
David O'Meara
Rancher
Posts: 13459
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
wrap it in a BufferedWriter is the first step

PrintWriter out
= new PrintWriter(new BufferedWriter(new FileWriter("foo.out")));
 
batuk chatuk
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks!!!

File f = new File("foo.xml");
PrintWriter pw = new PrintWriter(new BufferedWriter(new FileWriter(f)));
pw.write(xmlConverter.getXML(obj));

Using BW does speed up the process by almost 40%, although the PC remains in a numb state during that time..but it does work..

I m afraid this solution is not gonna be enough as there are around 100k entries on Production env at max.. ( I checked above sol. on 50000 records..)

Is there any possibility that i split the records and then in chunks i add them in "same file". (dont want to create more than 1 XML for one page..)

Thanks again...
 
batuk chatuk
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Its Done!!!

File writing is happening totally with all data at a time in above code..
So this is the answer to better and memory friendly code...

xmlSource = xmlConverter.convert(obj);
BufferedReader bufferedReader = new BufferedReader(xmlSource.getReader());
PrintWriter pw = new PrintWriter(new BufferedWriter(new FileWriter(f)));
int count = 0;
int buffer=1024;
char[] c1=new char[buffer];
while((count = bufferedReader.read(c1, 0, buffer)) != -1){
pw.write(c1, 0, count);
}
pw.flush();
pw.close();
 
batuk chatuk
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
For 100K records or more the above code didnt work on my machine.. ( i had kept JVM Heap size to Max. 1300K.. )
The reason was the XMLConverter was not able to convert that much data into XML So i wrote the code to divide all the ArrayLists in chunks and then convert them and write them into the same file..

FileWriter fw = new FileWriter(new File("foo.xml , true);

the flag in the constructor is for Appending the new data to same file..
and so file can be generated without any trouble..

Best Regards
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!