Which design pattern to use ?

[Prasanna KumarBP]

Hi,
Problem: My back-end application need to receive an XML data from an external system (over HTTP), that need to be updated in the database ( in the EJB layer). The size of an XML data is huge(>50 MB).
Example: Recieve existing "Bank Account" information in an XML format and create/udpate the same in the database.
Which design pattern can I use ? This is very urgent.
Thanks in advance.
- Prasanna

[Lasse Koskela]

"Prasannabp",
Your display name does not comply with our naming policy. Please change it here.
Thank you.

[Gurumurthy Ramamurthy]

Value List Handler Pattern.
Explanation:
If you want to receive a huge data, then consider DAO instead of EJB. The Value List Handler Pattern gets the list of objects from DAO and sends the clients a small quantities of data multiple times.
Cheers,
Guru

[Ilja Preuss]

Basically, there is not *one* design pattern you should use. Design patterns are typically applied at a much finer granule - you might be using a whole bunch of patterns for this problem, or none.

[Stan James]

Going off topic ... out of a 50 meg XML input, how much is tags and meta-data? How much is real data? I'd take a moment to step back and ask if XML is really the right choice. Course, if that's what somebody sends you, that's what you have to accept!

[Frank Carver]

50MB is not so big these days. Personally I'd parse the XML into a simple array or list representation in memory, then update the DB all in one go using whatever native bulk-update mechanism the database vendor provides. This will likely be orders of magnitude faster than messing with EJBs, as well as much simpler code.
Check out the web for help on your doing this with your specific database.

[Ilja Preuss]

Originally posted by Frank Carver:
50MB is not so big these days.

Our Ant-logfiles for full builds are bigger than that!

Personally I'd parse the XML into a simple array or list representation in memory, then update the DB all in one go using whatever native bulk-update mechanism the database vendor provides.

For a modern database, that might actually even be XML itself!

[HS Thomas]

For a modern database, that might actually even be XML itself!

Illja, I have read that you can generate a database from XML but not that the database is XML. I was under the impression that XML should only be used for mark-up .
What is an XML Database ? I guess you'd be able to capture details like super classes and other things OO than you can with a traditional database.
Microsoft's is developing a language (?) called X# for relational data access among other things.
Unifying Tables, Objects and Documents
[ September 08, 2003: Message edited by: HS Thomas ]

[Frank Carver]

Illja, I have read that you can generate a database from XML but not that the database is XML. I was under the impression that XML should only be used for mark-up .
I think what Ilja was getting at is that some databases will allow you to do bulk update operations by passing an XML representation of the data to the database. Usually this just means that the database has some sort of XML parser as well as an SQL parser built in to it.

[Ilja Preuss]

Originally posted by Frank Carver:
Illja, I have read that you can generate a database from XML but not that the database is XML. I was under the impression that XML should only be used for mark-up .
I think what Ilja was getting at is that some databases will allow you to do bulk update operations by passing an XML representation of the data to the database. Usually this just means that the database has some sort of XML parser as well as an SQL parser built in to it.

Yes, exactly.

[Robert Martin]

One idea: Store the 50MB in a flat file, and store the filename in the EJB layer.