• Post Reply Bookmark Topic Watch Topic
  • New Topic

Streaming nested XML file

 
ranbir singh
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,

What are possible approaches for streaming xml documents?
I have requirement where is the huge no. (in lacks) of payments information is provided in XML format (nested sructure)
and we need to parse and manipulate some fields and finally display the result in webapplication.

Any suggestion will be of great help.

Regards,
Ranbir
 
Greg Charles
Sheriff
Posts: 3010
12
Firefox Browser IntelliJ IDE Java Mac Ruby
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If the changes you're making are relatively small, the best approach may be to use a SAX parser. It has the advantage that you never have to have the whole file in memory at the same time. It's also relatively simple to use. You just have to write handlers for start tags, end tags, text, etc. What it's not good at is random access to elements in the document.

For random access, a better choice is DOM, which reads the entire XML document and builds up a tree of elements, which can be accessed or manipulated as needed. A newer option, JAXB, converts the XML document into a Java object structure. (There are also several open source products that do the same thing.) The big disadvantage is memory. The whole document must be represented in memory, not just a bit at a time.

Finally, StAX was made to sort of split the difference. It's not as memory hungry as DOM or JAXB, but it's supposedly easier to use than SAX, especially if you need to make significant changes to the XML elements as part of your processing. I say "supposedly" because I've never used it myself. Maybe someone else can weigh in on it.
 
ranbir singh
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Greg: Thanks for your inputs.
But the challange here is we are dealing with industry wideused payment formats and the payments volume in each file will be very high. Moreover, we need to do calculations/transformation on the input data before writting it to database.
 
Greg Charles
Sheriff
Posts: 3010
12
Firefox Browser IntelliJ IDE Java Mac Ruby
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Both SAX and StAX parsers allow you to do transformations on XML files without the need read the entire file in memory at once. Can you explain why that doesn't meet your challenge?
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!