• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Bear Bibeault
  • Jeanne Boyarsky
  • Tim Cooke
Sheriffs:
  • Knute Snortum
  • Junilu Lacar
  • Devaka Cooray
Saloon Keepers:
  • Ganesh Patekar
  • Tim Moores
  • Carey Brown
  • Stephan van Hulst
  • salvin francis
Bartenders:
  • Ron McLeod
  • Frits Walraven
  • Pete Letkeman

Object Stream for large volume of data  RSS feed

 
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Friends,

I have to write a service between DB and other applications.

This service is need to read the database and provide the stream of objects.
There will be millions of records in the DB.

I thought of below given approach.
I need to convert the records into Objects stream (Is is possible to do soo??!).
This stream will be send acros other applications which uses this object for further processing (like converting into flat file etc).

Please suggest me. How to impplement it.

Thanks in advance,
Bala
 
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I would not even attempt to do this with ObjectStream - instead I would attempt to represent each data item in some compact text format, for example XML. There are plenty of tools for converting between XML and Java objects.

Advantages:
1. You can test with text files that can be edited, passed around easily, saved as test cases with documentation as XML comments.
2. On the client side you can use well understood SAX or STaX parsing of XML
3. Clients don't have to have the same version of Java -OR- even written in Java, ever so flexible.

You may find my survey article on "pipeline" style processing of data to be helpful. Here is the link for part 2 of that article.

If you went with ObjectStream you would have to create a new connection for each object (or possibly collection of objects) and close it to finish the transmission correctly. With a text stream formatted as XML things are much more flexible.

Bill
 
Bala Thiruppathy
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks a lot for you genuine answer
 
Don't get me started about those stupid light bulbs.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!