Win a copy of Testing JavaScript Applications this week in the HTML Pages with CSS and JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Bear Bibeault
  • Ron McLeod
  • Jeanne Boyarsky
  • Paul Clapham
Sheriffs:
  • Tim Cooke
  • Liutauras Vilda
  • Junilu Lacar
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • fred rosenberger
  • salvin francis
Bartenders:
  • Piet Souris
  • Frits Walraven
  • Carey Brown

Editing big xml files

 
Ranch Hand
Posts: 290
Oracle Tomcat Server Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I have an xml file of 30M. I use the following code to delete the tags with no childs. But jdom simply does not build the document and
with out any exception the application goes out of this method.

The same code works fine for smaller file e.g. 9MBs?

My question.

1. Is there any file size limit?
2. What can be the solution to this?
3. What is the best way to remove tags with no childs from xml files?

Many thanks
 
Greenhorn
Posts: 17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
try launching application with -Xmx512m option
 
Ranch Hand
Posts: 223
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Similar thing in my project, the input files are tremendously huge, so we have written a program called as xmlfilesplitter which splits the large xml file into many smaller files(you can have a limit of say 20 records per file) and use that as input wherever you want.
To do this, you can use the opening tag name, say <item> and closing tag name </item> or whatever as the start and end point.
[ May 26, 2007: Message edited by: v ray ]
 
Ranch Hand
Posts: 2308
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Though I have not worked much on STAX (Streaming API for XML ; JAXP 1.3 onwards).You might consider.

Stax will give you the advantage of SAX (low on memory) and feature of DOM (editing/creation of XML) with which you can read and construct another xml on the fly without the nodes that you do not want.

Experts ; please suggest ?
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This might be a job for "pipeline" style XML processing. This article mentions some uses of XML pipeline toolkits. The ServingXML open source toolkit would be the one I would look at first.

Bill
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
    Bookmark Topic Watch Topic
  • New Topic