What to use when Applying XSL to 150Mb XML file

[raje malhotras]

Hi
I have a serious problem
I am converting very large XML(greater than 150 Mb) to another XML by using XSLT transformation in Java 5.
But in this process jvm uses a very large amount of memory(heap of jvm).

I have use XALAN and java inbuilt parsers for this transformation but still memory usage is very high ....

So i am searching for some parsers or something like that..... by which when i do this transformation my memory usage will be as low as posssible.........

so is there any way to do my memory usage minimum.....

Thanks for any help

[William Brogden]

If there is any way you can pre-process the monster XML file to remove unused data or re-arrange into a more logical order it will reduce the XSLT memory requirement.

How complex is the XSLT logic?

Bill

[raje malhotras]

XSLT logic is not complex .
It just repaces tag names and something like that.

The main problem is size of XML Because of which memory requirement is too high.......

Any suggestions are also invited.......

[raje malhotras]

There is no scope of pre processing the XML file .If i do so ,in that case the size of XML is still greater than 150Mb.

XSLT logic is not complex .
It just repaces tag names and something like that.

The main problem is size of XML Because of which memory requirement is too high.......

But i wanted to reduce memory requirement..............
Any suggestions are also invited.......

[Paul Clapham]

It doesn't matter how complex or simple your XSLT logic is, an XSLT processor will almost always load the entire XML document into memory before applying that logic. So your best bet is really to not get involved in designs that include giant XML documents. Not if you have to apply XSLT to them, that is.

However I believe that the SAXON transformer does apply some optimizations for XPath expressions that can be proven to be (I forget the exact term) serial. This means that you can apply the expression to a document while reading the document from beginning to end without having to back up, and it means that the transformer doesn't have to build a document tree in memory.

You might look into this, although I predict that as soon as you decide to use it, somebody will right away change the XSLT to something else simple where this optimization can't be applied, and you are right back to where you started.

Good luck.

[Amit Arora]

Hi

You could try for STX to be used for XML transformation. It does not load the document in the memory. The heap size would be less. But it is a different language used for transformation.