• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Distributed / Shared memory ?

 
Chaminda Amarasinghe
Ranch Hand
Posts: 413
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Hi All,

Im really curious how to implement a shared memory in clustered environment.

Let take an example, There are many ejb instances running on same jvm as well as different jvms. each instance does some kind of batch processing for a particular portion of data (lets say transaction details of a customer). To avoid 2 ejb instance are operating on same data (same customer) I think I want some kind of centralized shared memory to store currently operating data (customer), since some ejb instances are in different jvms. But Im not sure how to implement this kind of things.

Your help would be highly appreciates.

PS. Database is not an option

Thanks
 
Chaminda Amarasinghe
Ranch Hand
Posts: 413
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Is my question clear?
 
Ashwin Pai
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Chaminda Amarasinghe wrote:
Hi All,

Im really curious how to implement a shared memory in clustered environment.

Let take an example, There are many ejb instances running on same jvm as well as different jvms. each instance does some kind of batch processing for a particular portion of data (lets say transaction details of a customer). To avoid 2 ejb instance are operating on same data (same customer) I think I want some kind of centralized shared memory to store currently operating data (customer), since some ejb instances are in different jvms. But Im not sure how to implement this kind of things.

Your help would be highly appreciates.

PS. Database is not an option

Thanks


I think what you are talking about is the Mediator pattern (GoF)...
Mediator may be mplemented as an ejb. This will be responsible of keeping track of data that is getting processed and communicates appropritely to the collegues..

HTH
Ashwin
 
J J Wright
Ranch Hand
Posts: 254
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm not sure Mediator is the right approach here. From your initial question I'm assuming your trying to perform load balancing on the various tasks that make up your batch process. If the tasks are independent of each other you'd be better of encapsulating them in a command object and publishing them to a load balanced JMS queue. This way you share the processing among instances in your cluster and remove the need for any complex coordination or communication.
 
Ashwin Pai
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jonathan Aotearoa wrote:I'm not sure Mediator is the right approach here. From your initial question I'm assuming your trying to perform load balancing on the various tasks that make up your batch process. If the tasks are independent of each other you'd be better of encapsulating them in a command object and publishing them to a load balanced JMS queue. This way you share the processing among instances in your cluster and remove the need for any complex coordination or communication.


You are correct if the tasks are independent. That wasnt clear for me from the post. I thought, when he said multiple ejb instances, they were different ejbs working on different portion of the data (Customer) and wanted to share the info across the ejbs. I missed the fact that he was trying to avoid multiple ejb instances working on the same Customer, which would mean that using a LB JMS queue with the ejbs/mdbs processing this data would be the right approach. Using a JMS would ensure that the objects are processed by single ejb instance.

Sorry for the confusion..:-)

Ashwin
 
Chaminda Amarasinghe
Ranch Hand
Posts: 413
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Jonathan Aotearoa and Ashwin Pai,

Thank you very much for your valuable comments. Actually I have no idea how to implement a LB JMS queue. But for the now I use a JMS queue for this. May be that is a LB Queue. .

I used a singleton scheduler to generate a message for each portion of data (each Customer) and MDB instances pick messages appropriately (load balanced way) and do the process. scheduler shoots messages in very 5 mins (business rule).

Everything works fine in normal situations. But is some cases depending on the load of Customer's data, The time to process for a Customer exceeds 5 mins. So next cycle of scheduler fires a message for same Customer again and a new mdb instance starts operating on same Customer. That means 2 mdb instance are operating on same Customer. finally everything messed up due to this Race Condition.

I want to avoid that. Anyhow I want to track currently processing Customers. If a Customer is still processing, either scheduler should not fire a message for this user or MDB should filter out this Customer.

For the now system is still on single app server. So I use a simple ConcurrentMap to keep the track of currently operating Customers. But this will not work in clustered env.

Your help greatly appreciate

Thanks

 
J J Wright
Ranch Hand
Posts: 254
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
OK, I see your problem. If you want to stay with the theme of JMS you could approach this using a kind of sequencer, for want of a better term. For example:

1. You place all your customer processing commands in a single queue.
2. These commands are consumed by a single consumer. When a message is consumed the component maintains a reference to the customer id before publishing it to a second queue.
3. The second queue has many competing consumers, this is the load balanced queue (see app server cluster documentation about this).
4. When a competing consumer finishes processing a customer's data is sends a control message (via another queue) to the sequencer.
5. The sequencer deletes its reference to the customer.
6. Only messages relating to customers not currently referenced by the sequencer are placed in the second queue.

This way you can still process many customers concurrently but at the same time ensure sequential processing on a per customer basis. How you implement message storage in the sequencer is up to you. You could store messages in a map keyed on customer id or you could just put them back in the initial queue.
 
Chaminda Amarasinghe
Ranch Hand
Posts: 413
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Jonathan

I will try
 
toni bautista
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Chaminda Amarasinghe wrote:
Hi All,

Im really curious how to implement a shared memory in clustered environment.

Let take an example, There are many ejb instances running on same jvm as well as different jvms. each instance does some kind of batch processing for a particular portion of data (lets say transaction details of a customer). To avoid 2 ejb instance are operating on same data (same customer) I think I want some kind of centralized shared memory to store currently operating data (customer), since some ejb instances are in different jvms. But Im not sure how to implement this kind of things.

Your help would be highly appreciates.

PS. Database is not an option

Thanks



read this:

http://unserializableone.blogspot.com/2008/02/pod-racing-how-to-synchronize-threads.html




 
Chaminda Amarasinghe
Ranch Hand
Posts: 413
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Thanks toni bautista,

This is what I was looking
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic