I'd like to share with the community my latest development - the Object Store .
I built it because I couldn't find a solution which would allow me safely concurrently and efficiently work with large collections of in-memory objects. My scenario was:
- millions of objects,
- a hefty multi-core machine with 8+ Gb of memory available for the JVM
- about a hundred of threads accessing the collection, mostly reads
- I needed "indexing" on put so I could quickly remove objects matching certain criteria without iterating over the entire collection.
In such scenario "traditional" maps collections guarded by old-fashioned synchronized blocks fell short. The object store addresses these shortages by using ReadWriteLocks, indexes and concurrent processing of queries/updates.
If you face scenarios similar to mine - give the object store a try!
Best regards, Pavel.
Given the requirements it seems like java.util.concurrent.ConcurrentHashMap would be a good fit.
Could you specify the shortcomings of that standard Map implementation in your use case, and how your implementation differs to address those shortcomings?
Edit: Wuh?! What happend to the orginal post?
Edit++: Oh, seems like I replied mid-transfer to Blatant Advertising, nevermind...
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
ConcurrentHashMap is a map which doesn't throw exceptions during concurrent access, but some operations are unpredictable, according to its Javadoc. Also in my scenario I'd have to have, in addition to a map instance:
- A lock to provide consistency.
- Secondary maps for indexes and code to update them within the same write lock.
I am not sure what do you mean by indexing here -- a hash should be designed to spread out the elements evenly, and ideally, no more than one or two elements per bucket. I can't imagine any "index" that can improve the speed to find an element in a bucket of only one or two elements.
As for locking, the concurrent hash map, is "tricked out". It uses reader-writer locks to allow parallel reads. It uses segmentation to allow somewhat parallel writes. It is written using the atomic libraries to allow some optimistic locking, and better scaling. etc.
Forgot to mention... a few years ago, one of my former colleagues wrote a thread safe hashtable that is completely free of locks. And was able to scale it almost linearly up to 768 processors (the biggest JVM avail at that time). Really cool.
Object store is not a map. It is backed by a map or a collection and provides additional facilities for collection access and manipulation, like creating views and indexes, and executing queries and updates in multiple threads. You can take a look at the Javadoc or the documentation page to see the difference between Store and Map methods.
Best regards, Pavel.
I RELEASE YOU! (for now .... ) Feel free to peruse this tiny ad: