Win a copy of Java 9 Modularity: Patterns and Practices for Developing Maintainable Applications this week in the Java 9 forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Object Stream for large volume of data  RSS feed

 
Bala Thiruppathy
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Friends,

I have to write a service between DB and other applications.

This service is need to read the database and provide the stream of objects.
There will be millions of records in the DB.

I thought of below given approach.
I need to convert the records into Objects stream (Is is possible to do soo??!).
This stream will be send acros other applications which uses this object for further processing (like converting into flat file etc).

Please suggest me. How to impplement it.

Thanks in advance,
Bala
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I would not even attempt to do this with ObjectStream - instead I would attempt to represent each data item in some compact text format, for example XML. There are plenty of tools for converting between XML and Java objects.

Advantages:
1. You can test with text files that can be edited, passed around easily, saved as test cases with documentation as XML comments.
2. On the client side you can use well understood SAX or STaX parsing of XML
3. Clients don't have to have the same version of Java -OR- even written in Java, ever so flexible.

You may find my survey article on "pipeline" style processing of data to be helpful. Here is the link for part 2 of that article.

If you went with ObjectStream you would have to create a new connection for each object (or possibly collection of objects) and close it to finish the transmission correctly. With a text stream formatted as XML things are much more flexible.

Bill
 
Bala Thiruppathy
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks a lot for you genuine answer
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!