Win a copy of liveProject: Protecting User Data with Spring Security and OAuth2 this week in the Spring forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • paul wheaton
  • Devaka Cooray
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Liutauras Vilda
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
  • Piet Souris
Bartenders:
  • salvin francis
  • Mikalai Zaikin
  • Himai Minh

Which design pattern to use ?

 
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
Problem: My back-end application need to receive an XML data from an external system (over HTTP), that need to be updated in the database ( in the EJB layer). The size of an XML data is huge(>50 MB).
Example: Recieve existing "Bank Account" information in an XML format and create/udpate the same in the database.
Which design pattern can I use ? This is very urgent.
Thanks in advance.
- Prasanna
 
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
"Prasannabp",
Your display name does not comply with our naming policy. Please change it here.
Thank you.
 
Ranch Hand
Posts: 273
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Value List Handler Pattern.
Explanation:
If you want to receive a huge data, then consider DAO instead of EJB. The Value List Handler Pattern gets the list of objects from DAO and sends the clients a small quantities of data multiple times.
Cheers,
Guru
 
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Basically, there is not *one* design pattern you should use. Design patterns are typically applied at a much finer granule - you might be using a whole bunch of patterns for this problem, or none.
 
(instanceof Sidekick)
Posts: 8791
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Going off topic ... out of a 50 meg XML input, how much is tags and meta-data? How much is real data? I'd take a moment to step back and ask if XML is really the right choice. Course, if that's what somebody sends you, that's what you have to accept!
 
Sheriff
Posts: 6920
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
50MB is not so big these days. Personally I'd parse the XML into a simple array or list representation in memory, then update the DB all in one go using whatever native bulk-update mechanism the database vendor provides. This will likely be orders of magnitude faster than messing with EJBs, as well as much simpler code.
Check out the web for help on your doing this with your specific database.
 
Ilja Preuss
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Frank Carver:
50MB is not so big these days.


Our Ant-logfiles for full builds are bigger than that!

Personally I'd parse the XML into a simple array or list representation in memory, then update the DB all in one go using whatever native bulk-update mechanism the database vendor provides.


For a modern database, that might actually even be XML itself!
 
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

For a modern database, that might actually even be XML itself!


Illja, I have read that you can generate a database from XML but not that the database is XML. I was under the impression that XML should only be used for mark-up .
What is an XML Database ? I guess you'd be able to capture details like super classes and other things OO than you can with a traditional database.
Microsoft's is developing a language (?) called X# for relational data access among other things.
Unifying Tables, Objects and Documents
[ September 08, 2003: Message edited by: HS Thomas ]
 
Frank Carver
Sheriff
Posts: 6920
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Illja, I have read that you can generate a database from XML but not that the database is XML. I was under the impression that XML should only be used for mark-up .
I think what Ilja was getting at is that some databases will allow you to do bulk update operations by passing an XML representation of the data to the database. Usually this just means that the database has some sort of XML parser as well as an SQL parser built in to it.
 
Ilja Preuss
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Frank Carver:
Illja, I have read that you can generate a database from XML but not that the database is XML. I was under the impression that XML should only be used for mark-up .
I think what Ilja was getting at is that some databases will allow you to do bulk update operations by passing an XML representation of the data to the database. Usually this just means that the database has some sort of XML parser as well as an SQL parser built in to it.


Yes, exactly.
 
Author
Posts: 76
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
One idea: Store the 50MB in a flat file, and store the filename in the EJB layer.
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic