Jim:
My current thinking is to have regional caches to improve read performance and force writes to be asynchronous on the client side, but I'd like to avoid forcing async writes if possible.
If a write is converted to async then how is the result of the write (success/failure) reported to the user?
There will arise cases where the data becomes out of date by the time it is written to the server. This may lead to data corruption i.e. an older data over writes a newer data.
I am not so sure why the performance numbers you have given are inclusive of the network latency. I think the numbers must be for the server i.e. the persistence process and not the roundtrip time back to the client.
Using a cache for reading is good and more professional but doing an async write is a bit dicey.