• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Architecture Ideas Required - Distributed Persistence Service

 
Greenhorn
Posts: 17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi all,

I need to build a service which is used for persistence of arbitrary data. Typically this may be structured data saved out from spreadsheets...we don't care what the data structure is, or even the format (may be json, xml, yaml etc).
The service must be available over HTTP and have a WebDAV interface.
The service must also send out update messages over JMS for persisted data (inserts and updates). Update messages must be consistent with the state of the database (i.e. the last update message sent must be consistent with the database in the database).
We aim to complete small writes in under 100ms and reads in under 50ms.

Our current service is based on Slide, with an Oracle database, and is deployed in London only. The main problem currently is that the WAN link between Asia and London is awful (250ms round trip), so cross-region performance is not acceptable.
All data from all regions must be available globally, as it is read/written from many regions.

Any recommendations on suitable architectures, open source components etc to achieve these requirements much appreciated.
My current thinking is to have regional caches to improve read performance and force writes to be asynchronous on the client side, but I'd like to avoid forcing async writes if possible.

Thanks

Jim
 
Bartender
Posts: 1638
IntelliJ IDE MySQL Database Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jim:
My current thinking is to have regional caches to improve read performance and force writes to be asynchronous on the client side, but I'd like to avoid forcing async writes if possible.



If a write is converted to async then how is the result of the write (success/failure) reported to the user?
There will arise cases where the data becomes out of date by the time it is written to the server. This may lead to data corruption i.e. an older data over writes a newer data.

I am not so sure why the performance numbers you have given are inclusive of the network latency. I think the numbers must be for the server i.e. the persistence process and not the roundtrip time back to the client.
Using a cache for reading is good and more professional but doing an async write is a bit dicey.
 
Ranch Hand
Posts: 126
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Why don't you use a distributed cache with write-behind technic. Fist level updates can be made directly into the cache itself. and the write behind technic to update to the database. this way sice you are wriing into the cache you should have a good latency.

Products like Tangosol and Gigaspaces will let you distribute geographically

Thanks
Raees
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic