After having spent some time investigating message queues, I have a slight feeling that they may not be what I am looking for.
Now, in desperation, I ask for some advice.
I want to maintain a queue and have the following requirements:
1. Messages in the queue must be durable so if the queue goes down, they will still be available when it comes back up.
Thus also not too big. I must run the queue inside a standalone application (no container).
3. I must be able to remove arbitrary message from the queue.
4. I must be able to modify one or more messages in the queue.
5. Should perform well.
This is not the most important requirement, but nevertheless nice to have.
6. I will only have one single producer and one single consumer.
7. Messages must be consumed in the order in which they are produced, that is FIFO.
I have been looking at JBoss HornetQ, which I must say is a very nice message queue. The only problem seems to be 3 and 4.
I have also been thinking about using a cache, which is persisted in some way (vague ideas).
If anyone has any thoughts or pointers I would be very grateful. I really do not want to reinvent the wheel.
Thanks in advance!
posted 10 years ago
Given that items in the queue each have a unique key, I am contemplating combining a map with the queue. Both the producer and consumer will have access to the map - the producer checks for duplicates and if there are none, inserts the item into the queue and also inserts a corresponding entry into the map. The consumer retrieves an item from the queue, then uses the map to determine if the item has been deleted or modified. If the item has been marked as deleted, then no processing takes place. If the item has been marked as modified, then the data of the item from the queue is modified before the item is processed.
This would, as far as I think, allow me to:
- Mark an element in the queue as deleted by modifying its entry in the map.
- Modify an element in the queue by entering updated information into its entry in the map.
- Determine if there is a duplicate entry in the queue by using the map.
All the above operations can be done in O(1) using the map and there is no need to browse the queue.
I would, of course, have to persist the map too, but I feel that persisting a single map object is to be preferred before implementing a solution to my problems entirely from scratch. Also, I feel they queue implementation I am investigating performs persistence far better than I am able to do.
#3 - you can partially achieve using MessageSelectors.
#4 - is a No as far as I know. The only way to do that is to post a message back with the modified payload/header. You can probably use a correlation id to link it to the original message.
Message for you sir! I think it is a tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop