XML vs. ???

[Andrew Shafer]

I have a bunch of data that I need to store and look up later. Now the structure of the data is not conducive to storing in your conventional relational database. It's too much to store in one XML file, but XML seems like one good way to model the data and the XML files could be related to search parameters and stored as large objects in a database. Seems like a good idea.
Now I thought of another way to solve the problem and I was hoping some smart ranchers could give their opinions about either approach.
Instead of storing xml files, why not model the data with Java, then store the data as serialized java objects. The only difference being, storing XML vs. Serialized Java in the database.
Anyone see any pro or con to either approach?

[Guillaume Compagnon]

Serialized Java objects are not human readable.
and marshalling in serialization makes the size of data growing perhaps more than putting XML tags over your data.
If you want to store Java objects and retrieve them quickly(faster than with XML DOM) I advice U to hava a look to JDO (Java Data Objects), this is a specification for the mapping between Objects and DB.
there is the open source project <A HREF="http://castor.exolab.org/">Castor</A> that implements the JDO spec ... Craig Russell is the JDO Specification Lead
Or if You use EJB/J2EE, just use the entity beans...
the last product, I see, is Java Blend from Sun (http://www.sun.com/software/javablend/index.html) but it seems to be no more supported by Sun.

[W. Scott Means]

The decision about when to store data in XML and when to store it in another format is definitely a tough one. In most cases, XML is a _great_ transport mechanism for moving data from one platform to another. For instance, enabling your mainframe to export information in XML format and then importing it into a web server for transformation and display is a great use of XML. Storing information that doesn't fit neatly into a relational database schema is another case. For instance, if you have a hierarchical data structure that doesn't translate to a table/join structure, you might want to store that as XML in a database column.
But an XML document makes a poor database. It's large, slow, and you can't (easily) do complex joins on the information in it. For very small applications where you might have been tempted to do a flat-file or ISAM database, XML might be right. But in general relational data belongs in the database, unstructured content belongs in XML.
As for serializing Java objects via XML, there have been several efforts in that direction. Yes, using the native Java serialization format is more compact. But it's also a closed loop. You won't be able to take that BLOB and feed it to an XSLT processor and display a nice web page. Or take it and send it to a mainframe that will extract some information during a batch cycle.
So if you know that the data will never leave your closed system, by all means use the most compact and speediest means to store information. But if you even suspect that the content will be useful to another system, or you might want to build a web service around it, make it available in XML. It doesn't cost that much, and makes life much easier down the road.
------------------
W. Scott Means
author, Strategic XML
[e-mail]
[This message has been edited by W. Scott Means (edited October 17, 2001).]

[timothy zimmerman]

I am not sure if this would be an appropriate solution but here it is anyway. You say the information is too large to store in one XML file yet you think that you may gain some use out of storing it that way? Would it be possible to break that one XML file into smaller pieces and then assemble them into one file using references to the smaller documents?

[Andrew Shafer]

Thanks for the input, but I'm still not quite convinced either way.
Ok, let me break it down so you have a little better idea what I need, then maybe you will have more specific advice.
The database needs to warehouse kinetic and thermodynamic data. There will definitely be a web based interface.
The user will search the database for reactions, the elementary reactions contain from 2-4 molecular compounds. This is all mapped quite nicely into a relational database. Now, each compound has structural information and thermodynamic information, that doesn't map well to a relational database. (Ever heard of a Hessian matrix?) Furthermore, the thermodynamics will have different values for the same species, based on what methods were used to make the calculation. From the thermodynamics of all the involved compounds, kinetic calculations will be available. Each reaction may have over a hundred different values for the thermodynamics and kinectics, based on the methods used to generate the values (empirical, semi-empirical, theoretical, level of theory, etc.) So for each reaction, there needs to be information on 2-4 compounds, a transition state, possibly a minumum energy path, and then the kinetics. Fun huh?
Now, I'm quite sold on the idea of using XML to transport data, even to store data. My question though, if I'm going to create java objects from the data, what would be the trade off of either approach. I suspect the serialized java files would be smaller.
Its easy enough to implement some sort of .toXML() methods that can generate files or streams from the objects for presentation or transportation. Its also easy enough to extract the fields from XML to populate object. I guess my big uncertainty is, where will I get better performance? How does processing XML compare, time wise, to serializing/deserializing java objects?
Any thoughts?

[raj sekhar]

Guess what? I have a similar question though with a different flavor...! I have posted this on a different thread..but since the discussion is here, I want to ask you guys again !
We port data to mainframe but they expect data on a flat file(linear data)...
****************************
Hi guys
I am a novice to XML and its architecture.In our project we collect lot of data from the user(web interface) and save it on a session and finally, we write all the data as a linear string of CLOB in to oracle8i. This may not be a very wise thing to do but it served our purpose. Now with, XML, we'd like to structure things and convert them into xml and save the xml in the DB. In this case we will need to xtract the data everytime before passing it to another system(we pass it as linear data).
Now, any of you guys see sense in going to xml and do we have any benefits of using xml in this scenario
OR
For our need, is it better to do what we are doing currently?
Thanks
****************************8
Please let me know if the question is not clear !
Thanks
Raj

[Andrew Shafer]

Raj,
If you are just going to convert a string to XML, store that as a CLOB, retrieve the CLOB, then convert it back into the original string to send to the client, then XML doesn't make much sense.

If you can use the data in any other applications or present it to anyone else beside the original client, then, in my opinion, XML starts to have benefits for you.
Did everyone else run off?
Going back to my original question, here is a bit more information that might help you to convince me either way. The database doesn't exist in a vacuum. There are executables that make new calculations based on old calculations. For example, A + B -> C + D might be in the data, but someone wants values for A + X -> C + Y, they could get the stored data for A and C, supply thermodynamics for X and Y at the same level of theory A and C were calculated and get kinetic calculations. Another application might take a collection of reactions and all the kinetic info then run simulations over time starting with certain reactants. One goal is to automate the creation of input files for this application. Which basically brings me back to the original question, why would I choose XML transforms over building the files from the same data modeled in java objects? Coding one or the other doesn't seem to have much advantage either way, writing the XSL doesn't appear to have an advantage over writing pure java methods. Does anyone believe there would be a considerable performance difference either way?
regards,
Andrew

[timothy zimmerman]

I am still here just following the discussion, not sure I have any more information to help. I am just watching the thread and learning.
I would just add this, maybe it's valid maybe not but I think it boils down to the data and what the real purpose of the data is. If you are really looking only to store and exchange information/data I would maybe lean more toward XML. If you are looking for something more active you may want to create objects which can perform those actions and use the data to populate those objects.

[Guillaume Compagnon]

Originally posted by Andrew Shafer:

Its easy enough to implement some sort of .toXML() methods that can generate files or streams from the objects for presentation or transportation. Its also easy enough to extract the fields from XML to populate object. I guess my big uncertainty is, where will I get better performance? How does processing XML compare, time wise, to serializing/deserializing java objects?
Any thoughts?

What you said makes me think about a tool I ve used one year ago.
this is Breeze XML Studio http://www.breezefactor.com/
this tool brings a XML file (just a sample of your data) and generate Java class that could be use to make calculations, ... and this tool generates a .toXML() method.
if that could be useful ...the best thing to do is to try