Locking strategy URLyBird 1.1.1 with ReentrantLock

[rinke hoekstra]

After having read threads about locking on this forum for several weeks, doubting, reading threads again, etc, etc, I eventually had the guts to create my own locking strategy and lock method.

Please consider my method, and please see my questions below it.

General Strategy:

Data class implements given interface by sun, and is a Facade delegating tasks to two singleton classes: DataAccess and LockManager.

In DataAccess all methods which do read or write from the database have a synchronized block inside them (Create, Read, update, delete, find).

LockManager has a private ReentrantLock, a private Condition, and a HashMap with Integers (recordnumber) and Longs (the cookie). The HashMap is created via Collections.synchronizedMap.
I'm not using any data caching.

The lock method looks like this:

    public long lock(int recNo) throws RecordNotFoundException { 
        ...
        LockManager.lock.lock();
        try {
            while(cookieMap.containsKey(record)) {
                LockManager.lockReleased.await();
                ... //1
            }
            ... //check if exists, throw RecordNotFoundException if not
            ... //add to the map
            //2
            return cookie;
        } finally {
            LockManager.lock.unlock();
        }
    }

Questions on this:

General: Is this strategy OK you think?

//1: the call to lock.await normally is inside a try / catch block for an InterruptedException. I'm doubting what to put inside this catch clause.

//2: Note that there is no call to signalAll(). This is only done at unlocking in the unlock method. Is that sound?

3: I'm having doubts between using ReentrantLock or just using a synchronized block in stead of it, synchronizing on the RandomAccessFile object. Apparantly the code doesn't change much because of such a change. I was wondering what are peoples arguments for one or another. My own thinking would be that ReentrantLock is a bit more clear for novice programmers.

4: For a completely differnt approach: I saw someone mentioning "lockable object pattern", but I couldn't find much on the net about it. Anyone knows if this is a good approach, and how it works??


Thanks in advance.
[ June 10, 2007: Message edited by: rinke hoekstra ]

[Mandy Bowman]

i have no idea about this. As far as I can see this seems too complicated to me. I always just use a synchonrised block. i have no idea about how to do these things with reentrant locks and all that. what does everyone else thing is it better to use the old way of doing it or use these new locking classes?

[John Stone]

I think it is OK. I guess, that you are also reading SCJD by Andrew M., because your stragey looks almost like the one described in book.

1, perhaps throw some run-time exception, that some error occured
2, sounds good to me
3, I would not synchronize on RandomAccessFile in lock method, I think this may lower performance, if you want to synchronize on something then on cookieMap (I'll probably go with ReentrantLock)
4, I'm just guessing this is situation, where you are not waiting on cookieMap (or one static lock) but on specific record ("Multiple notification objects" chapter in SCJD by Andrew M.) (Every record has its own lock, if I understand it well)

[Mihai Radulescu]

Hi,

It look ok to me. You only need to justify why you choose to use the new java.util.concurent features and not the clasical synchronized.
I, personal I use the classical the synchronize, this was fitting with my requirements.

Why you synchronize on RandomAccessFile ? (point 3 in your post).

Regards,
M

[rinke hoekstra]

Thank you all for replying.

@John Stone: Yes, I have this book too. But it also looks to me like the most simple approach.

About the multiple notification objects: I read that too, but concluded it was beyond the scope of the assignment. But I think that is something different than "Lockable Object Pattern". See also this thread: https://coderanch.com/t/187461/java-developer-SCJD/certification/Lockable-objects-client-IDs - I found it and was intrigued by it, but couldn't find anything about it on the net.


@John Stone, @Mihai: yes, synchronizing on the RandomAccessFile would not be optimal, I realize that now. Better is on the Map.

Thanks, Rinke

[rinke hoekstra]

Originally posted by Mandy Bowman:
i have no idea about this. As far as I can see this seems too complicated to me. I always just use a synchonrised block. i have no idea about how to do these things with reentrant locks and all that. what does everyone else thing is it better to use the old way of doing it or use these new locking classes?

Hi Mandy,

ReentrantLock is in fact not very different from synchronized. The main difference is that ReentrantLock works with an explicit condition.

In synchronized, if there is some reason why the thread should not continue, you just call wait, usually inside a while loop.

In the ReentrantLock, you can do the same, but then you don't use a while loop usually, but you use a Condition Object.

So ReentrantLock is basically almost the same as synchronized, only it is a bit more object oriented, because the condition on which you would check in a synchronized block has now been made into an object.

[Ken Boyd]

Originally posted by rinke hoekstra:
After having read threads about locking on this forum for several weeks, doubting, reading threads again, etc, etc, I eventually had the guts to create my own locking strategy and lock method.

Please consider my method, and please see my questions below it.

General Strategy:

Data class implements given interface by sun, and is a Facade delegating tasks to two singleton classes: DataAccess and LockManager.

In DataAccess all methods which do read or write from the database have a synchronized block inside them (Create, Read, update, delete, find).

LockManager has a private ReentrantLock, a private Condition, and a HashMap with Integers (recordnumber) and Longs (the cookie). The HashMap is created via Collections.synchronizedMap.
I'm not using any data caching.

The lock method looks like this:

    public long lock(int recNo) throws RecordNotFoundException { 
        ...
        LockManager.lock.lock();
        try {
            while(cookieMap.containsKey(record)) {
                LockManager.lockReleased.await();
                ... //1
            }
            ... //check if exists, throw RecordNotFoundException if not
            ... //add to the map
            //2
            return cookie;
        } finally {
            LockManager.lock.unlock();
        }
    }

Questions on this:

General: Is this strategy OK you think?

//1: the call to lock.await normally is inside a try / catch block for an InterruptedException. I'm doubting what to put inside this catch clause.

//2: Note that there is no call to signalAll(). This is only done at unlocking in the unlock method. Is that sound?

3: I'm having doubts between using ReentrantLock or just using a synchronized block in stead of it, synchronizing on the RandomAccessFile object. Apparantly the code doesn't change much because of such a change. I was wondering what are peoples arguments for one or another. My own thinking would be that ReentrantLock is a bit more clear for novice programmers.

4: For a completely differnt approach: I saw someone mentioning "lockable object pattern", but I couldn't find much on the net about it. Anyone knows if this is a good approach, and how it works??


Thanks in advance.

[ June 10, 2007: Message edited by: rinke hoekstra ]

Rinke,
I am with you 100%. In my assignment I have used ReentrantLock as it is clean and simple to understand compare to classic synchronization. I am not saying synchronization is difficult but SUN won't come up with something new if old was good enough. Andrew mention in his book about short coming of earlier version where multiple notification won't be possible. But after reading Andrew's book I like ReentrantLock as it is simple, clean and best for my solution. I have also added methods and made changes to Andrew's code.

Cheers,
Ken

[Dimirti Slyus]

Hi ! My first post here...

... //check if exists, throw RecordNotFoundException if not
... //add to the map

The code is from your LockManager class, right ?
My question is: how are you going to check "check if exists" ? will you have a reference to your FileAccess class from LockManager class ? or I got something wrong here ... Thanks

[Gabriel Vargas]

I agree with Rinke and Ken (thanks for your argument Ken, i'm thinking in a justification for use new lock classes and it looks good ) and i also implement a DatabaseManager and a LockManager.

Dimirti

My question is: how are you going to check "check if exists" ? will you have a reference to your FileAccess class from LockManager class ? or I got something wrong here ... Thanks

If i'll have a reference to FileAccess the objetive to separate functionality of data interface in two classes is lost. LockManager only locks records using a map. In data implementation i use DatabaseManager to verify than record exists, then i use LockManager to made lock/unlock operation then these classes has their own responsabilites.

[Ken Boyd]

Originally posted by Dimirti Slyus:
Hi ! My first post here...



The code is from your LockManager class, right ?
My question is: how are you going to check "check if exists" ? will you have a reference to your FileAccess class from LockManager class ? or I got something wrong here ... Thanks

actually you don't have any access to FileAccess class from LockManager to " check if exists". Basically you separate FileAccess, Lockmanager and Data class separate but flow will be from data to fileaccess to lockmanager.

So you will call lock from data to lockmanager after you get lock check for exists if yes go ahead with your operation i.e. update/delete or create otherwise throw recordnotfoundexception

Ken

[rinke hoekstra]

Dimitri brings up an issue that I'm wrestling with too. I think it is not so easy, or maybe I'm missing things?

In fact, we have to check if the record exists, and this must be done from inside the synchronized (or locked) code, otherwise it makes no sense because another thread could creep in and spoil it all.

So what Ken and Gabriel are suggesting is something like this (if I understand it well):

Data class implements DB, and has a lock method.

1 - starts with ReentrantLock.lock (meaning that the ReentrantLock object is in the Data class and not in LockManager???)

2 - this lock method calls LockManager.lock,
3 - LockManager.lock checks if Map contains the record,
-> if yes, wait.
-> if no, return

4 - then Data checks FileAccess if the record exists,

5 - then Data calls LockManager.add (or something like that) to add the record to the map

6 - and then the Data.lock calls ReentrantLock.unlock

It seems to me that this is pretty complicated, as a DataAccess call is right in the center in between all kind of stuff which should belong in the LockManager.

It would be much more simple if the (singleton) constructor of LockManager is passed a reference to FileAccess, so FileAccess is always available at its use, and the simple check if the record exists can be done from within the LockManager.lock method without this up and down going.
Would this really spoil "the objective of separating the functionality of the two classes", as Gabriel puts this?

Or is it even better to conclude that locking functionality and Data access functionality are so intermingled, that the most simple sollution is just NOT to separate them and put them all simply in the Data class, which then stops being a Facade???

Or am I thinking too complicated this late at night, and am I missing something completely???

[Gabriel Vargas]

Hi Rinke,

Sorry if i cann't explain myself, english isn't my native language.

There are three main ideas than i'm thinking, the Data class functionality, DatabaseManager functionality and LockManager functionality. I explain how i implement this functionalities in my project.

First DatabaseManager class functionality is concerned to provide all data read/write file operations like a DAO.

Second LockManager class provides functionality to verify if a record is logical locked, this is achieved verifying on a map than contains records thna are locked (it doesnt know than a DatabaseManager exists, it only know the lock map). If record are locked is used wait/notify of standard synchronization and later notifyAll (you can see than you must sinchronize on a object I personally sinchronize on the lock map).

At last Data class calls DatabaseManager and LockManager methods, for example if you call find method of data class this method calls find mehtod of DatabaseManager and have no calls to methods of LockManager class, but if you call lock method then it first call read method of DatabaseManager, then call lock method of LockManager.

As you can see there are a sinchronization over the lock map of the LockManager class, but there are neccesary another sinchronization over DatabaseManager, it can be achieved over the same DatabaseManager methods or as i implement over the DatabaseManager object of the Data class. I use standar sinchronization mechanism over objects than i don't need to call a wait/notify mechanism (in the Data class), and new lock classes when i need use wait/notify (methods are now await/signalAll than seems to me more clear to use).

As i read in Andrew's book Data class provide data access and lock mechanims, mean of "the objective of separating the functionality of the two classes" is than Data acts as a Facade of DatabaseManager and LockManager an this class do their operations whitout know each other as i try to explain. I hope it helps you.

Regards.

[Ken Boyd]

Originally posted by rinke hoekstra:
Dimitri brings up an issue that I'm wrestling with too. I think it is not so easy, or maybe I'm missing things?

In fact, we have to check if the record exists, and this must be done from inside the synchronized (or locked) code, otherwise it makes no sense because another thread could creep in and spoil it all.

So what Ken and Gabriel are suggesting is something like this (if I understand it well):

Data class implements DB, and has a lock method.

1 - starts with ReentrantLock.lock (meaning that the ReentrantLock object is in the Data class and not in LockManager???)

2 - this lock method calls LockManager.lock,
3 - LockManager.lock checks if Map contains the record,
-> if yes, wait.
-> if no, return

4 - then Data checks FileAccess if the record exists,

5 - then Data calls LockManager.add (or something like that) to add the record to the map

6 - and then the Data.lock calls ReentrantLock.unlock

It seems to me that this is pretty complicated, as a DataAccess call is right in the center in between all kind of stuff which should belong in the LockManager.

It would be much more simple if the (singleton) constructor of LockManager is passed a reference to FileAccess, so FileAccess is always available at its use, and the simple check if the record exists can be done from within the LockManager.lock method without this up and down going.
Would this really spoil "the objective of separating the functionality of the two classes", as Gabriel puts this?

Or is it even better to conclude that locking functionality and Data access functionality are so intermingled, that the most simple sollution is just NOT to separate them and put them all simply in the Data class, which then stops being a Facade???

Or am I thinking too complicated this late at night, and am I missing something completely???

I think you miss something completely.. here it goes

1. Data class has lock method --> it will simply call LockManager to lock the record no matter it is deleted, booked or ready for booking no matter what because that is the function of lock method to lock the record.

2. Now once you get LockCookie from LockManager saying record is locked and record number is in map (it might be deleted record we don't care at this point)

3. As we know client call lock method to do three things at the most create/update or delete the record. In step 2 we locked deleted record but at the time we have no idea what client wants to do with deleted record. It might want to create new record at the same record number since we have to reuse deleted entry to create new one.

4. In case client call delete method after locking the deleted record you throw recordnotfoundexception

5. In case client call create method after locking the deleted record you go through all records to get deleted entry which can be used to create new record or create new record at the end of the file if lock record is not deleted and we find no deleted entry from database.

6. In case client call update method after locking the deleted record you throw recordnotfoundexception since it is deleted means not found according to spec..

did I miss something...

Ken

[rinke hoekstra]

Hi Ken,

thanks for your reply.

OK, something is getting very clear at the moment...

But first to find out if we understand each other well:

I'm talking about the implementation of the DB.java interface method lock, which is declared in the interface as follows:

// Locks a record so that it can only be updated or deleted by this client.
// Returned value is a cookie that must be used when the record is unlocked,
// updated, or deleted. ....
  public long lock(int recNo) throws RecordNotFoundException;

This lock method should return a cookie value which is then used by unlock, delete, or update.
Of course, this lock method has no notion of what will be the following action: (delete, update, etc??). The client passes the returned cookie value into the next method (delete, update), but the lock itself has no notion of this.

So what you are basically saying is that this lock method (of the Data class, but also of the LockManager class) just ignores the RecordNotFoundException it is bound to raise, as it just does not check at all if the record exists?
Because the delete, update, whatever method which follows will raise the RecordNotFoundException anyway if they find out that the record is not existing?

At first, this sounded to me as a violation of the interface contract of lock, as this suggests that RecordNotFoundException should be thrown. Also, ANY other method (delete, read, etc) is checking this condition (that the record exists in the database) when throwing a RecordNotFoundException - so ingoring it here seems an inconsistancy.

However, I just realize that this whole stuff with RecordNotFoundExceptions in the lock method is really nonsense anyway. If we first call lock, and check on the record if it exists, and then call delete (or some other method), and then again in this method check if it exists - that's overdoing it quite a bit, and really only one of those two checks is well enough. As soon as the record is locked, no other thread could access it anyway.

Then, having an unlock method throwing a RecordNotFoundException is possibly even more nonsense.

So, if I get it right, your sollution is:
just ignore the RecordNotFoundException in the lock method, as it is bullshit anyway, and put this motivation in choices.txt. And DON't check if the record exists inside the lock method.

And the same counts for unlock.

In case of deciding ignoring the RecordNotFoundException is too risky (as it is declared in the interface contract): this would be the alternative:

Data.lock method does:
1) call LockManager.lock, which logically locks rec (meaning: puts it in the map, even if the rec does not exists)
2) LockManager.lock returns to Data.lock
3) Data.lock checks if rec exists
4) If not, call LockManager.unlock first for logically unlocking, and then raise RecordNotFoundException.
5) if rec does exist, just return the cookie.


Leaves me with the question: what did others decide in this? Do you think it is too much of a risk to just ignore RecordNotFoundException in lock/unlock, even though in fact it is completely unnecessary to check it inside these methods?
[ June 30, 2007: Message edited by: rinke hoekstra ]

[rinke hoekstra]

Thanks Gabriel also for your reply. Above I reacted on Ken's posting, but your reaction helped me as well in getting things more clear.

[Ken Boyd]

Originally posted by rinke hoekstra:
[QB]So, if I get it right, your sollution is:
just ignore the RecordNotFoundException in the lock method, as it is bullshit anyway, and put this motivation in choices.txt. And DON't check if the record exists inside the lock method.

If you don't allow to lock deleted record entry then how you will do this

// Creates a new record in the database (possibly reusing a
// deleted entry). Inserts the given data, and returns the record
// number of the new record.

e.g. Record number 11 is deleted

in order to reuse deleted entry i.e. record 11 you have to lock that record. Now if you don't lock it two threads can access it and you are fail without question for allowing two clients to have fun with one record.

Now the question of throwing recordnotfoundexception in lock/unlock method I throw them if record number passed is invalid not that they are deleted. I mean consider hypothetical situation in direct test of Data class and some client pass invalid record number i.e. -1000 or something like that will never exist for locking so Recordnotfoundexception thrown and same client also go for update or delete and unlock with invalid cookie. In that case you throw RNFE in unlock and update or delete methods.

One more thing to add. Client GUI has no Create button so it will never reuse deleted entry (i.e. record 11). So GUI will never pass deleted record since it will be caught and thrown as error message in front end.

What you say..

Ken
[ June 30, 2007: Message edited by: Ken Boyd ]

[rinke hoekstra]

Hi Ken,

thanks for your reply.

If you don't allow to lock deleted record entry then how you will do this

quote:
// Creates a new record in the database (possibly reusing a
// deleted entry). Inserts the given data, and returns the record
// number of the new record.

In fact, I don't reuse deleted entries. I've seen quite a lot of people ignoring this, even some having 100% scores. Disk space is definitely not a limited resource nowadays, and it will be very unlikely that this simple database will suffer from insufficient available disk space because we do not reuse deleted recs.
As the create method has its own synchronized block in which any new record is just appended at the end of the datafile, this saves you from all kind of locking problems when trying to reuse an existing deleted record, so it makes life really simpler - exactly according to the specification to "keep it as simple as possible".

Now the question of throwing recordnotfoundexception in lock/unlock method I throw them if record number passed is invalid not that they are deleted. I mean consider hypothetical situation in direct test of Data class and some client pass invalid record number i.e. -1000 or something like that will never exist for locking so Recordnotfoundexception thrown and same client also go for update or delete and unlock with invalid cookie. In that case you throw RNFE in unlock and update or delete methods.

Yes, I can see that point. But strictly seen, this check would not even be necessary as I argued above, as the next called method in line (delete, update) will perform this check too, and throw that exception too.
Would it go to far if I'm saying that we are basically throwing the RecordNotFoundException in lock / unlock just for the sake of throwing it because it is in the interface contract?

[Ken Boyd]

Originally posted by rinke hoekstra:
Hi Ken,

thanks for your reply.




Yes, I can see that point. But strictly seen, this check would not even be necessary as I argued above, as the next called method in line (delete, update) will perform this check too, and throw that exception too.
Would it go to far if I'm saying that we are basically throwing the RecordNotFoundException in lock / unlock just for the sake of throwing it because it is in the interface contract?

I think you try to implement everything from DB interface but throwing RNFE is not must condition but we still throw in best possible manner.

one more scenario where client select record to delete it and sequence will be lock()--> delete()--> unlock() but hey unlock has to throw RNFE since record is deleted (poor client will get RNFE for no reason). In that many ranchers called unlock from delete method not pretty but hey everybody has own way of doing things.

I mean interface provide you general outline and you take it from there but lock()--> update/delete/create (reuse entry)-->unlock()--> has to follow that sequence to be consistent across the board.

In my implementation of create method I find deleted record and lock them after that check if they are still deleted if not go back in loop to find deleted entry, once found create record at same location and return record number. If none found create new one at the end of file (syn db so no two threads can create at same time)

You are leaving something out to implement something so go to choice.txt

uhhhhh.. thanks for your reply.

Ken
[ July 01, 2007: Message edited by: Ken Boyd ]

[Dimirti Slyus]

Hi guys! sorry I was not here for couple of days.
Here is my lock method of the Data class:

public long lock(int recNo) throws RecordNotFoundException {
long cookie = lockManager.lock(recNo);
boolean recordExist = database.exist(recNo);
if (recordExist) {
return cookie;
}
else {
lockManager.unlock(recNo, cookie);
throw new RecordNotFoundException();
}

}

where:
lockManager.lock(recNo) generates random cookie number, adds the entry into the map or waits if the record is locked.
database.exist(recNo) just checks if record exsists and not deleted.

do you see any potential holes in here ? Thanks

[Ken Boyd]

Originally posted by Dimirti Slyus:
Hi guys! sorry I was not here for couple of days.
Here is my lock method of the Data class:

public long lock(int recNo) throws RecordNotFoundException {
long cookie = lockManager.lock(recNo);
boolean recordExist = database.exist(recNo);
if (recordExist) {
return cookie;
}
else {
lockManager.unlock(recNo, cookie);
throw new RecordNotFoundException();
}

}

where:
lockManager.lock(recNo) generates random cookie number, adds the entry into the map or waits if the record is locked.
database.exist(recNo) just checks if record exsists and not deleted.

do you see any potential holes in here ? Thanks

how you will create new record by reusing deleted entry? as of now you are not allowing to lock deleted record. If you don't lock deleted record (e.g. 11) two threads can create new record at same record i.e. 11 .

[Dimirti Slyus]

how you will create new record by reusing deleted entry? as of now you are not allowing to lock deleted record. If you don't lock deleted record (e.g. 11) two threads can create new record at same record i.e. 11 .

Thanks for your post, Ken.

Agree but creating the record does not require locking. I will explain.
Let's say CLIENT_1 calls Data.create method.
The Data class will call database.create method
My database class has cached deleted records. So the database.create will:
1. Lock the cache.
2. check if there is deleted record in cache. If it is it will use the recNom for writing the record. If not it will use the last record +1.
3. synchronize on the database and write the new record.
4. delete entry from deleted records cache.
5. Unlock the cache and return recNo.

CLIENT_2 will try to create record by calling the same Data.create that will call database.create.
Now it will stop waiting for the lock to be released from the 1st step. After cache is released it will not contain deleted record used by the first client. Therefore this client will use another record or new one - step 2.

Did I miss anything ?

[Ken Boyd]

Originally posted by Dimirti Slyus:


Thanks for your post, Ken.

Agree but creating the record does not require locking. I will explain.
Let's say CLIENT_1 calls Data.create method.
The Data class will call database.create method
My database class has cached deleted records. So the database.create will:
1. Lock the cache.
2. check if there is deleted record in cache. If it is it will use the recNom for writing the record. If not it will use the last record +1.
3. synchronize on the database and write the new record.
4. delete entry from deleted records cache.
5. Unlock the cache and return recNo.

CLIENT_2 will try to create record by calling the same Data.create that will call database.create.
Now it will stop waiting for the lock to be released from the 1st step. After cache is released it will not contain deleted record used by the first client. Therefore this client will use another record or new one - step 2.

Did I miss anything ?

My design have no cache..so it is simple if you want record for any operation follow lock()-->read()->>update/delete/create-->unlock()..

recordnotfoundexception thrown in lock and unlock method if record number is not valid. unlock throw securityexcetion if given cookie doesn't match with record number.

Ken
[ July 20, 2007: Message edited by: Ken Boyd ]

[Ziji (Jay) Zhang]

Dimirti,

I am thinking of doing the similar approach for create method,
If you keep a list of deleted record,
there are two issues that I am worried about
1. you are facing nested locking, in create, you first lock the deleted Record List, then you lock the data file.
2. in deleted method you again will do nested locking, add deleted
record into the list.


Ziji

[Ziji (Jay) Zhang]

Dimirti,

I am thinking of doing the similar approach for create method,
If you keep a list of deleted record,
there are two issues that I am worried about
1. you are facing nested locking, in create, you first lock the deleted Record List, then you lock the data file.
2. in deleted method you again will do nested locking, add deleted
record into the list.


Ziji

[Ziji (Jay) Zhang]

Dimirti,

I am thinking of doing the similar approach for create method,
If you keep a list of deleted record,
there are two issues that I am worried about
1. you are facing nested locking, in create, you first lock the deleted Record List, then you lock the data file.
2. in deleted method you again will do nested locking, add deleted
record into the list.


Ziji

[Liviu Carausu]

Hi Ziji, Dimitri,
To avoid the nested locking problem, why not synchronize on the database(RAF) also when updating the cache with deleted records ? The operations must be anyway atomic:
1) at create: reading the next free entry - writing the record content in DB - removing the record from cache
2) at delete: marking the record as deleted in db - adding the deleted record index into cache.
I hope that helps... Your question also helped me because I discovered the nested monitors....
Thanks,
Liviu

Originally posted by Ziji Zhang:
Dimirti,

I am thinking of doing the similar approach for create method,
If you keep a list of deleted record,
there are two issues that I am worried about
1. you are facing nested locking, in create, you first lock the deleted Record List, then you lock the data file.
2. in deleted method you again will do nested locking, add deleted
record into the list.


Ziji