• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Part 2: persistent messages vs database storage

 
Claudiu Dumitrescu
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I have a scenario like this: an object will become available to SuD after some number of days after its creation. Taking in consideration that any loss of object means loss of money for the user, which of these two approaches do you think is more appropriate:

1) use a JMS Queue to be read every some period, to select the messages which are due to become available to the system and physically create them in database.

2) Or store them in the database right away after creation and use a scheduler to make them available by creating the necessary DB rows when the time is due

Second solution I think will pollute my DB's tables in some way since the domain model, as it was given, does not seems support this. And also will force me to a more complex management of domain objects even for simple scenarios.

On the other hand, the first solution rise the question of how reliable I should think the Message Broker System is. If this fails, I will loose all the objects which is unacceptable.
 
ntumba lobo
Ranch Hand
Posts: 180
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Usually if you do not want to lose any messages between two systems it is recommended to use a messaging solution
that will give you guaranteed devlivery.
So I would go for option 1 in your case
 
Rishi Shehrawat
Ranch Hand
Posts: 218
Hibernate Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Keeping message in queue & processing it later in my opinion is not the correct use of messaging. Normally you consume messages as they come. Holding messages on queue might not be a very efficient. The consumer will have to traverses messages to figure out which ones to consume. How will this work with message driven beans which are triggered when a message arrives in the queue? You will have to tweak the normal behaviour of message consumption, which normally is not recommended.

In terms of processing logic for record creation, it should be more or less same, no matter where it is triggered from. In my opinion using database along with a batch job that runs once a day (or more depending on your requirement) will be a better, cleaner approach. Read messages & write them in the database. The message can be just dumped in the database without much processing. The batch job/scheduler can then create records. This scenario is how we typically use batch jobs/scheduler.
 
ntumba lobo
Ranch Hand
Posts: 180
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am surprised by this statement
"Keeping message in queue & processing it later in my opinion is not the correct use of messaging"
Is it not the actual definition of asynchronous processing where producer/consumer don't to be available at the same time ?

As for
"Normally you consume messages as they come"
I would say it depends on your requirement. Actually the consumer consumes the message when it is ready to do so.
Unless you set one, there is no time limit for a message on a queue.

MDB is not the only option to consume messages as you correctly said they get triggered as soon as a message arrives.
So I would not use it here.
You can also have a synchronous consumer that just reads the queue when it is ready to consume the message.

So really I would stick to option 1 as JMS provides all the insfrastructure to meet the requirement simply.

 
Rishi Shehrawat
Ranch Hand
Posts: 218
Hibernate Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Keeping message in queue to be processed after a few "days", to me is not correct definition of asynchronous processing. Yes one of the major advantages of asynchronous processing is that it can cater to scenarios where consumer is down, having read the initial problem statement this does not seem to be the case. The consumer is available, but the message has some time based processing to be done. This is the reason the message is being retained in the queue for days.

MDB provides a number of advantages over other approaches of message processing. MDB were designed to cater to the normal use case for asynchronous processing. Not being able to use MDB perhaps is a pointer to the fact that you are not using JMS for the correct requirement.



 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 35279
384
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I agree with Rishi. A queue is not meant to be a long term storage location.
 
ntumba lobo
Ranch Hand
Posts: 180
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Interesting I seem to be the only one to think of the created object as a TEMPORARY data that must stored somewhere
before it can be processed and that is of no use to anything else during the storage.
This really sounds like the field of queues to me. The temo data just happens to span a couple of days. no biggie.
DB is more for long term storage data that needs to be processed later on in the DB (searched, updated...).
It doesnt seem to be the case here.

I cant help but think that the DB approach implies some development and runtime overhead.
We dont know how complex the object is, it may not be so trivial to put into a DB and then rebuild it from the DB.
Then you need play very carefully with your transactions to mimic the guaranteed delivery mechanism provided by jms.
All this integration code, storing object in DB, implementing scheduler, build object back from DB, then delete it from DB.
All this need to be custom developed, then tested whereas you can get it all for free with jms and you can put your object straight into a jms message.

Classic debate anyway.
Claudiu you have my view and the one from the others, you'll get to choose eventually.

 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 35279
384
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think of a JMS queue as a way to transfer data between two applications one of which might not be up right now. And I think of a regular (Java) queue as a first in first out structure. I know JMS has selectors that let you filter by time. I can't help but think that behind the scenes it goes through the whole queue to find the matches. If I was using this approach in a real world application, I'd have to stress test it. Also, I would lose the ability to monitor the queue to make sure messages aren't getting backlogged since it would be normal to have a pile of messages on the queue.

I think of that as the difference. The word "storage" doesn't appear anywhere in my mental model of a JMS queue.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic