Bodgitt: Locking requirements

[Derek Canaan]

Hi,
I'm doing the Bodgitt and Scarper assignment with lockRecord method returning a long cookie and i'm unsure about the locking requirements:
1. If user runs the application in "alone" mode, can the application bypass the lock -> read -> update -> unlock protocol?
2. Do i need to take into consideration this scenario:
RMI server is started, and is serving some remote clients, and the administrator of the computer hosting the rmi server starts another process and run the application in "alone" mode. In this case, if the "alone" mode bypasses the locking protocol, then the db file could be corrupted by concurrent access. If so, then the "alone" mode must also implement the lock-> read -> update -> unlock protocol. Is this right?
Any help on these is appreciated.
Thanks
Derek

[Bharat Ruparel]

Hello Derek,
You asked the question in your point 1 above and pretty much answered it in point 2.
I did my assignment so that the same code base for the data server is being used both in the alone mode and the network mode. Besides accounting for the situation that you describe in your point 2 above, there is a practical side of it too: it is much easier to develop your application in the alone mode first since you don't need to worry about the implementation of the network functionality (whether it is RMI or socket powered) at all. You make sure that all your read and write operations are working correctly, your locking scheme, in theory, works ok, your GUI does everything that has been asked for in the assignment. Once you do that, you can tackle the network functionality much more confidently. I designed and developed my application that way. I am new to RMI, and I didn't really need additional uncertainity which will inevitably be introduced if I started with RMI right away. Having the application working more or less correctly in the alone mode first (since you can't really test the locking scheme thoroughly until you implement the network functionality and have multiple remote clients simulteneously accessing your server) gave me all kinds of confidence since I wasn't quite sure of what I was doing with RMI initially.
Now, it is quite possible to design your application so that you can pretty much plug-in the network piece in the application with the help of a couple of adapters and keep all your alone mode code unchanged! That is what I did (I had a lot of help doing that from people like Andrew, Mark, Jim, Max, Phil, Vlad, Tony, and a few others). But it seems to have worked out quite well.
All this is amounts to saying that you are proceeding in the right direction.
Welcome to JavaRanch.
Regards.
Bharat

[Philippe Maquet]

Hi Derek,
Welcome to this forum. Hope you'll enjoy it as I did and do.
I agree with all what Bharat just wrote :
1. Don't bypass the locking scheme in stand-alone mode.
2. The scenario you describe is right. But it's not the reason for which I advice you to not bypass the locking scheme in stand-alone mode (it wouldn't help anyway as it doesn't span multiple JVMs). You should have something like this in your instructions :

You may assume that at any moment, at most one program is accessing the database file; therefore your locking system only needs to be concerned with multiple concurrent clients of your server.

As the server and the stand-alone client are two different programs, don't care about possible conflicts.
Best,
Phil.

[Derek Canaan]

Hi Bharat and Philippe,
Thanks guys for the prompt replies. First, just wanna say scjd-starters like me have been truly enriched by guys like you for the unreserved sharing of knowledge and experience .

2. The scenario you describe is right. But it's not the reason for which I advice you to not bypass the locking scheme in stand-alone mode (it wouldn't help anyway as it doesn't span multiple JVMs). You should have something like this in your instructions :

quote:
--------------------------------------------------------------------------------
You may assume that at any moment, at most one program is accessing the database file; therefore your locking system only needs to be concerned with multiple concurrent clients of your server.
--------------------------------------------------------------------------------
As the server and the stand-alone client are two different programs, don't care about possible conflicts

Phil, as i understood your reply, this requirement DOES permit the "alone" mode to bypass the lock->read->update->unlock protocol because we shouldn't care for the scenario in point 2 as per requirement.
Secondly, the locking protocol would NOT help in scenario 2 anyway because in this case, 2 JVMs on the server will be competing for the same db file, and the locking protocol cannot resolve conflicts of this nature.
The chief reason then (as Bharat suggested) we should not bypass the protocol even for "alone" mode is so that the same base codes can be used for the "network" mode development even though the locking protocol is redundant in the "alone" mode because no other "programs" will be competing for access to the db file.
Are there any other reasons for not bypassing the locking protocol in "alone" mode apart from rendering it easier for later "network" mode development?
thanks in advance,
Derek

[Philippe Maquet]

Hi Derek,
Yes, you may bypass locking in stand-alone mode. But by doing so, you wouldn't gain anything IMO, and at the cost of a less maintainable code.

Are there any other reasons for not bypassing the locking protocol in "alone" mode apart from rendering it easier for later "network" mode development?

I don't see any other reason.
Best,
Phil.

[Derek Canaan]

Thanks Phil

[Derek Canaan]

Hi,
I have another question on locking design.
If i design my locking such that:
1. DBAdapterRemote -> Data
2. Data is a singleton
3. Read and write methods in Data has synchronized modifier (to prevent dirty reads)
In this case, do i still need to enforce the lock/update/unlock contract?
The reason why i thought it is not needed is as follow:
Suppose the DBAdapterRemote has a book method. In the book method, the adapter just delegates to the single data instance with a single method call: Data.updateRecord(long recNo, String[] data, long lockCookie) instead of:

      lockRecord
      updateRecord
      unlock

I am not worry about concurrent write-write (or concurrent read-write) on the same record, because theses requests will be queued as there is only one Data instance (singleton) and the read and write methods are synchronized.
I'm sure i'm missing something. Please advise,
Thanks,
Derek

[Michael Couck]

Hi Derek,
This is exactly my question. If the methods are synchronized, and they need to be because even if you sync a block inside the read / write methods, another thread could change the parameters before the first thread finishes it's processing, so you need to sync the whole method. So if you are syncronizing all the methods, like myself (more about why in a sec) then the current thread gets the lock on the Data object it's self (i.e. the implicit "this" object) then no other thread could enter any of the synchronized methods, leaving the locking mechanism completely redundant??? Of course if the interface wasn't so ugly, you could then in fact implement a locking at the record level, which would be elegant and more efficient. But as it stands there really is no way to make the locking beneficial at all! And I have been through this for a week already. Some of the locking schemes I have conjured up are suprisingly complex, and I still don't see any way around not locking the methods from the top and still have a benefit for the locking mechanism.
The reasom I lock the read methods is because if you don't other threads enter the method and corrupt your data, I tested this with 30 000 threads doing arbituary reads and this is in fact so, the data was inconsistant. I have seen that some guys are locking only a block inside the method, but I can see the holes in the logic, the second thread will change the parameter before the first uses it, am I missing somthing?

Michael

[Michael Couck]

Hi again,
I think I have a solution for my previous doubt . The only way that the locking mechanism could be beneficial is by NOT implementing the singleton model in Data, the locking table, or vector, whichever you use must be static and in the event of multiple Data instances, they will compete for the lock with a reference to the lock collection. Problem is that the spec specifically states the solution should be simple, easily understood by a junior programmer, however the final mechanism I have implemented is rather complicated.
I do have another question to pose to the community . I have changed the Data class, implementing book and cancel methods, so my reference now cannot be

 
DB data= new Data();

it has to be

 
Data data= new Data();

which defeates the purpose of an interface in the first place. Does anyone think it prudent to change the interface it's self, i.e. the DB (UrlyBird)?
One more question, has everyone implemented the create and delete methods? The spec says nothing about the GUI / user having the control over these functions, and logic dictates that this should not be controlled by the user for security reasons, so has everyone implemented these methods and not exposed them to the user?
Thanks
Michael

[Philippe Maquet]

Hi Derek and Michael,

Derek:
In this case (Phil: data is singleton and read/write methods are synchronized), do i still need to enforce the lock/update/unlock contract?

As you write yourself, it's a contract ... and a contract (the interface) *must be* fulfilled.

Michael:
If the methods are synchronized, and they need to be because even if you sync a block inside the read / write methods, another thread could change the parameters before the first thread finishes it's processing, so you need to sync the whole method.

I must disagree. Methods parameters are put on the stack (same as a local variable). So no other thread has access to them.

Michael:
Of course if the interface wasn't so ugly, you could then in fact implement a locking at the record level, which would be elegant and more efficient.

Some people here implemented locking at the record level without breaking the provided interface. Jim is one of them.
-----
If you synchronize all read/write Data methods on a single Data instance, of course the locking scheme may look redundant.
The real question is : do you really *need* to synchronize read/write methods as a whole ?
Best,
Phil.

[Michael Couck]

Hi Phil,

I must disagree. Methods parameters are put on the stack (same as a local variable). So no other thread has access to them.

Wow didn't know that the parameters are thread specific, that certainly sorts out some doubts! However I have found that local variables like Vector, are shared by the threads that enter the method. For example

1) Vector v= new Vector()
2) add data to vector - specific to thread 1
3) return vec to thread one for futher processing

If thread 1 gets sliced out at the end of line 2, and thread 2 then creates a new vector adds it's own data, then thread 1 again, returns the vector the data in the vector is thread 2's data. Do you mean local variables with references to primary data like "int", that they are specific to the thread that used them first? So for example

1) int i=0;
2) i += data;
3) return i;

Thread 1 enters, creates "i", adds it to data and stops.
Thread 2 enters, creates "i", and stops.
Thread 1 returns it's "i", "i" contains (i+data).
Is this the case?
Thanks Phil.
Michael

[Michael Couck]

Oh I have another question seeing that I am here. If an io exception is thrown inside a write method, the data could be partially written, i.e. corrupt, is it necessary to have a rollback mechanism? The answer to this is going to make or completely ruin my day.
Just use DB2 and have done with it , what do you think?
Michael

[Philippe Maquet]

Hi Michael,
Parameters in Java are passed by value, meaning that the called method receives a copy of the parameter passed. Their scope is as local as a local variable, and no other thread may access them. That's the same with parameters which reference objects on the heap, *BUT* that's the reference which is copied, not the object itself. So, while no other thread will have access to the reference passed as a parameter, other threads could access the object referenced, as far as they have access to another reference to it.
Put in other words, there is no risk at all with primitive parameters.
With object references, there is no risk either, as far as the object referenced is instantiated within the call chain.
Similar rules apply to returned values : no risk of concurrent access for primitives and for objects created in the call chain.
Hope it's clearer !

If an io exception is thrown inside a write method, the data could be partially written, i.e. corrupt, is it necessary to have a rollback mechanism? The answer to this is going to make or completely ruin my day.

Forget about any transactional mechanism ! If an IOException is thrown when accessing the file (read/write), the file is/may be corrupt, period. Providing some recovery system would go far beyond this assignment.
Best,
Phil.
[ December 06, 2003: Message edited by: Philippe Maquet ]

[Michael Couck]

Hi Phil,
Thanks that is much clearer(parameter access).
And thanks no rollback made my day! I was just about to sift through the rollback code in MySql to have a look at what they did.
See ya
Michael

[Philippe Maquet]

I was just about to sift through the rollback code in MySql to have a look at what they did.

[Derek Canaan]

Hi Michael and Phil,
I'm glad that you join in this discussion thread. Sorry that my turnaround response time is not so prompt as i'm in and out of this assignment due to other commitments.

[Michael]: I think I have a solution for my previous doubt. The only way that the locking mechanism could be beneficial is by NOT implementing the singleton model in Data, the locking table, or vector, whichever you use must be static and in the event of multiple Data instance.

But implementing singleton Data is so much simpler and my assignment specifically says:

A clear design, such as will be readily understood by junior programmers, will be preferred to a complex one, even if the complex one is a little more efficient.

Furthermore, my assigment has cookie in the lock signature, so the cookie can be used as the client identity. For assignments without cookies, it makes sense to have multiple Data instances as these instances can be used as the client's unique identity.
Michael, does your assignment has cookies in the lock signature?

[Phil]: The real question is : do you really *need* to synchronize read/write methods as a whole ?

Hi Phil, i synch on the read and write methods to prevent dirty read. Is there another way?
Or is dirty read acceptable as long as you document it that your design allows for dirty read. In this case, the client gui may get inconsistent data displayed. What happen if the client highlights the row with inconsistent data and click on book or update, would this jeopardise the integrity of the db file?

it's a contract ... and a contract (the interface) *must be* fulfilled

So if i implement Singleton Data instance and synch on my read and write methods, the using of lock/update/unlock is done just to fulfill the assignment requirements though it is redundant.
Please advise,
Derek

[Derek Canaan]

Hi,

[Michael]: So if you are syncronizing all the methods, like myself

If i implement singleton Data and synch on read and write methods, i do not need to synch on the lock and unlock methods because they are 'redundant' anyway - Redundant in the sense that they do not contribute anything extra to the concurrency issue (read-write or write-write) as all read or write requests are queued as Data is singleton and read and write methods have the synchronized modifier.

[Phil]: The real question is : do you really *need* to synchronize read/write methods as a whole?

If 90% of the code in the method needs to be in a synch block, then it will be simpler for junior programmers to understand the codes if i put the sync modifier on the method signature itself rather than using a synch block.
What do you think?
Derek