• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Complex high performance message queues

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have an extremely high performance high scale high available problem. It's sort of combining producer/consumer, broadcast/subscribe in a very high volume basis. Think 10k transactions per second with responses needed in 50msecs in a 24x7x365 up time scenario.

I have a pool of producers spread across multiple servers that get work. Ideally each producer would get a unit of work and broadcast the job out to all consumers of different types.
1. A Producer instance broadcasts job to consumer type A, consumer type B, consumer type C spread across multiple servers.
2. Consumer type A, B, and C work on the problem and respond with their individual results.
3. There will be many instances of consumer type A (or B or C), but only 1 instance should work on any one job.
4. Producer instance waits for results from A, B, and C and then picks the most optimal result from the work done by A, B, and C. If A, B, or C are slow to respond, they lose (expiry maybe?).
5. Producers and consumers will all be in multiple servers (VIPs).

What are the best technologies to consider for this?
 
Ranch Hand
Posts: 2187
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
MQ Series is the right tool for this. IBM has a talented group of software engineers. They would certainly be able to help you determine which technologies are best suited for the requirements and also how to design a solution that works.

Good luck!
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That really sounds like a JavaSpaces problem to me, given the transaction requirement.

GigaSpaces has both developer and commercial versions.

With that volume, will network speed be the limiting factor? Let us know what you find out!

Bill
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic