• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Jeanne Boyarsky
  • Ron McLeod
Sheriffs:
  • Paul Clapham
  • Liutauras Vilda
  • Devaka Cooray
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

NX: My Locking Mechanism

 
Ranch Hand
Posts: 697
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello all,
Here's my locking mechanism.
1. All read, update, delete, book operations on the db are done in the following way. As all these operations are record level, I used record level locking for these operations.
a. Synchronize lockedRec Vector
b. If the record is already locked, give up the cpu
c. If not, obtain the lock on the lockedVe Vector
d. Lock the record
e. Notify other threads on lockedRec Vector
e. Perform the operation
f. Synchronize lockedRec Vector and b...c
g. Unlock the record
2. For operations like search operations, i.e. findByName and other find operations and also for creating new record, am locking the db file
a. Synchronize on the lockedRec vector
b. Check for each record, if its already locked, give up the cpu
c. If record is not locked, lock the record
d. Repeat steps b..c till to get hold of all records and lock them
e. Perform find/create operations
f. Synchronize on the lockedRec and unlock the records(i.e. db file)
This is working fine. But, does this create problems that I cannot see? Am still testing... I am pretty much sure that this works , but again I maybe wrong and could'nt see things other can.
Please comment on the design. Thanks
 
Satish Avadhanam
Ranch Hand
Posts: 697
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello all, again.
As in my design I am sure that at any point of time only one thread is working on one record, I am using the record number itself as the cookie that is returned. This is fine, right? I mean the cookie concept in general is that it must be unique for each thread that is working on say a record. As I am doing record level locking, thought of using the record number itself as the cookie. Is it ok?
Thanks.
 
author and jackaroo
Posts: 12200
280
Mac IntelliJ IDE Firefox Browser Oracle C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Satish,
You are right - this will work. However you may have unnecessary processing happening.
Some questions for you:
  • Why are you locking the records for read operations? (There is one valid case for that - just want to see if it applies in your case)
  • Why are you notifying other threads in step 1e?
  • Why do you lock the entire database for a find operation? What does the find operation return to the calling class?


  • I am using the record number itself as the cookie that is returned. This is fine, right?


    You can do this.
    Many people are trying to add some (minimal) security / safety into their application by using non guessable cookies. With your scheme, there is nothing to stop me writing a program that tries to unlock a record I don't own the lock for - since I know what the cookie will be, it is easy for me to do.
    If you do use the record number as the cookie, then I would recommend you put a comment in your design documents stating that safety is not a design requirement. I would also recommend that you do not put in your API documentation the fact that you are returning the record number - that way a future modification could make a more safe solution without breaking documented functionality.
    Regards, Andrew
     
    Satish Avadhanam
    Ranch Hand
    Posts: 697
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Andrew,

    Originally posted by Andrew Monkhouse:
    Hi Satish,
    You are right - this will work. However you may have unnecessary processing happening.
    Some questions for you:

  • Why are you locking the records for read operations? (There is one valid case for that - just want to see if it applies in your case)


  • Andrew I am not using static RAF, so many threads will be writing, deleting, updating at the same time. The reason why I am locking db while reading is -- if I want to read record number 7, and if I'm not using locking, there might be another thread, which wants to delete the record or update the record. So while I'm reading, I'm giving room for another threads to change the data. For this reason, I'm locking records before reading. Is it correct? And you are saying about a valid case, is this the valid case or its a different scenario?
  • Why are you notifying other threads in step 1e?


  • Consider the psuedo code

    Lets assume t1, t2, t3, t4, t5 threads are present.
    t1, t2, t3 wants to operate on record number 100, t4 on 200, t5 on 300.
    Each thread has to lock the record number in lockedRecords before performing any function on that record. To do this, they have to obtain a lock on lockedRecords itself and then check if the record number is already locked. If the record is already locked they will wait by calling wait() on lockedRecords.
    Now coming to the example, if t1 gets a lock on lockedRecords, it checks if record 100 is already locked or not, if not it locks record 100 and returns from lock() method. Then reads and unlocks. Meanwhile, if t2 or t3 calls lock() and gets lock on lockedRecords, they see that record 100 is already locked. So they will wait by calling wait() on lockedRecords. But if t4 or t5 gets lock on lockedRecords, they will lock their records and goes on. notifyAll() is for threads t2, t3 which are waiting by calling wait() method. You think this is OK?
  • Why do you lock the entire database for a find operation? What does the find operation return to the calling class?


  • The same reason as why I'm locking while doing read operation. find operation returns the records which match the given criteria.


    Many people are trying to add some (minimal) security / safety into their application by using non guessable cookies. With your scheme, there is nothing to stop me writing a program that tries to unlock a record I don't own the lock for - since I know what the cookie will be, it is easy for me to do.
    If you do use the record number as the cookie, then I would recommend you put a comment in your design documents stating that safety is not a design requirement. I would also recommend that you do not put in your API documentation the fact that you are returning the record number - that way a future modification could make a more safe solution without breaking documented functionality.

    OK Andrew. First, I will try to change cookie to provide some security, if I can't I will document it accordingly. Thanks.

    Regards, Andrew


    Thanks Andrew. Please correct me if I'm wrong.
     
    Andrew Monkhouse
    author and jackaroo
    Posts: 12200
    280
    Mac IntelliJ IDE Firefox Browser Oracle C++ Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Satish,

    Originally posted by Andrew Monkhouse:
    1 Why are you locking the records for read operations? (There is one valid case for that - just want to see if it applies in your case)
    Originally posted by Satish Avadhanam:
    Andrew I am not using static RAF, so many threads will be writing, deleting, updating at the same time. The reason why I am locking db while reading is -- if I want to read record number 7, and if I'm not using locking, there might be another thread, which wants to delete the record or update the record. So while I'm reading, I'm giving room for another threads to change the data. For this reason, I'm locking records before reading. Is it correct? And you are saying about a valid case, is this the valid case or its a different scenario?


    So are you opening the RAF and closing it with each operation? For example, each call to read() will result in a new instance of the RAF being opened, read, and then closed?
    This is the scenario where you might want to consider record locking for reads.

    Originally posted by Andrew Monkhouse:
    2 Why are you notifying other threads in step 1e?
    Originally posted by Satish Avadhanam:
    Consider the psuedo code

    Lets assume t1, t2, t3, t4, t5 threads are present.
    t1, t2, t3 wants to operate on record number 100, t4 on 200, t5 on 300.
    Each thread has to lock the record number in lockedRecords before performing any function on that record. To do this, they have to obtain a lock on lockedRecords itself and then check if the record number is already locked. If the record is already locked they will wait by calling wait() on lockedRecords.
    Now coming to the example, if t1 gets a lock on lockedRecords, it checks if record 100 is already locked or not, if not it locks record 100 and returns from lock() method. Then reads and unlocks. Meanwhile, if t2 or t3 calls lock() and gets lock on lockedRecords, they see that record 100 is already locked. So they will wait by calling wait() on lockedRecords. But if t4 or t5 gets lock on lockedRecords, they will lock their records and goes on. notifyAll() is for threads t2, t3 which are waiting by calling wait() method. You think this is OK?


    The call to notifyAll() should be unnecessary. Consider a very simple synchronized block:

    If two threads (T1 & T2) try to call this synchronized block, the following will happen:
  • T1 and T2 are both in Runnable state
  • T1 gets the mutex on commonObject, and continues running
  • T2 attempts to get the mutex on commonObject, and goes into blocked state (not wait state)
  • T1 does whatever work it needs to do, then releases the mutex on commonObject. As soon as it does so, the JVM transitions any threads that were blocked waiting for commonObject into ReadyToRun state. There is no need for a call to notify() or notifyAll() - the JVM handles the state transition automagically.

  • Sooner or later the JVM checks which threads are in ReadyToRun state and finds T2 is ready to run, so it resumes T2 which then gets the mutex on commonObject and continues to run.

    Now, just in case you are unsure about this, here is a simple program that will show that threads do go into and out of blocked state without us doing anything.

    The method wasteSomeTime() does exactly what it?s name suggests: it takes some time to complete without really doing anything useful. It?s main purpose is just to spread out the operations over time so that we can see what is really happening. The important thing is that I never call sleep() or wait() in this (or anywhere else in this program), as all we want to look at is how the synchronization works.
    One of my runs of this program gave me the following output:

    Note that thread 1 was not the first thread to start, but it was the first thread to own the mutex - this is because the JVM thread scheduler had been swapping the threads around as they were running the wasteSomeTime() method.
    More importantly though, you should be able to see that although threads 0 and 2 were blocked, when thread 1 left the synchronized block, threads 0 and 2 were transitioned into ReadyToRun state, and one of them did get to run (and get the mutex on commonObject) before thread 1 finally finished running.
    So you can see that there is no need to call notify() or notifyAll() when you leave a synchronized block just because you are leaving the synchronized block.
    Now lets go back to your example, which I will make a bit simpler. Threads T1 and T2 are trying to lock record 100. Thread T3 is trying to lock record 200.
  • T1, T2, and T3 all try to get the mutex on lockedRecords
  • T1 gets the mutex, T2 and T3 go into Blocked state
  • T1 checks if record 100 is locked and then locks it
  • T1 calls notifyAll() (which has no effect)
  • T1 leaves synchronized block, which transitions T2 and T3 from Blocked to ReadyToRun state


  • The thread scheduler starts running T2
  • T2 gets the mutex, T3 goes into Blocked state
  • T2 checks if record 1 is locked, and goes into Wait state, which transitions T3 into ReadyToRun state


  • The thread scheduler starts running T3
  • T3 gets the mutex
  • T3 checks if record 200 is locked and then locks it
  • T3 calls notifyAll() which transitions T2 from Wait state to blocked state
  • T3 leaves synchronized block, which transitions T2 from Blocked to ReadyToRun state


  • The thread scheduler starts running T2
  • T2 gets the mutex
  • T2 checks if record 1 is locked, and goes into Wait state


  • That last step was unnecessary, and caused you to use more CPU cycles on the server than desirable for no benefit.
    Take out the call to notifyAll(), and what would happen:
  • T1, T2, and T3 all try to get the mutex on lockedRecords
  • T1 gets the mutex, T2 and T3 go into Blocked state
  • T1 checks if record 100 is locked and then locks it
  • T1 leaves synchronized block, which transitions T2 and T3 from Blocked to ReadyToRun state


  • The thread scheduler starts running T2
  • T2 gets the mutex, T3 goes into Blocked state
  • T2 checks if record 1 is locked, and goes into Wait state, which transitions T3 into ReadyToRun state


  • The thread scheduler starts running T3
  • T3 gets the mutex
  • T3 checks if record 200 is locked and then locks it
  • T3 leaves synchronized block


  • At the end of that run, T1 and T3 both own locks, and T2 is in wait state. Exactly what we wanted!

    Originally posted by Andrew Monkhouse:3 Why do you lock the entire database for a find operation? What does the find operation return to the calling class?
    Originally posted by Satish Avadhanam:
    The same reason as why I'm locking while doing read operation. find operation returns the records which match the given criteria.


    Personally I think that this is going to be slow and reduce concurrency, with little benefit. (Note that this is a personal opinion - you are welcome to disagree )
    You have said you are returning records, not record numbers.
    Lets say that you do a search which would ordinarily only return record 1 from the database, and another thread is concurrently trying to change record 1 so that it would no longer be returned by the search:
  • Thread 1 locks the entire database
  • Thread 2 waits for record 1 to become available
  • Thread 1 searches the entire database, finds record 1 matches
  • Thread 1 unlocks the entire database
  • Thread 2 modifies record 1
  • Thread 1 returns record number 1, which in the real database no longer matches (although it may still appear to match in thread 1's local copy of the record


  • So you have not gained anything by locking the entire database - you are still returning potentially dirty (dated) data.
    Try the same procedure without locking the entire database and you should find the same result - you are returning potentially dirty data.
    But if you don't lock the entire database, then two finds could be running concurrently. Or a find and a read, or a find and a write. Or any combinations of concurrent operations. We regain the concurrency that the instructions call for.
    Note that returning dirty data is acceptable (and almost impossible to avoid without sacrificing concurrency). You just have to verify the data when it comes time to book the record. (You have to decide what level of checking you are going to do when you do the booking, and justify that in your design decisions document.)

    Originally posted by Andrew Monkhouse:Many people are trying to add some (minimal) security / safety into their application by using non guessable cookies. With your scheme, there is nothing to stop me writing a program that tries to unlock a record I don't own the lock for - since I know what the cookie will be, it is easy for me to do.
    If you do use the record number as the cookie, then I would recommend you put a comment in your design documents stating that safety is not a design requirement. I would also recommend that you do not put in your API documentation the fact that you are returning the record number - that way a future modification could make a more safe solution without breaking documented functionality.
    Originally posted by Satish Avadhanam:
    OK Andrew. First, I will try to change cookie to provide some security, if I can't I will document it accordingly.


    Note that I did say that you could keep with your current design, as long as you document it.
    My personal preference is to try and make a more robust solution than is necessary (and normally Mark jumps in with a comment that simple designs are preferable to complex solutions).
    Regards, Andrew
     
    Satish Avadhanam
    Ranch Hand
    Posts: 697
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by Andrew Monkhouse:
    So are you opening the RAF and closing it with each operation? For example, each call to read() will result in a new instance of the RAF being opened, read, and then closed?
    This is the scenario where you might want to consider record locking for reads.


    Yes Andrew, I am opening the RAF and closing it for each read operation. That's why I am doing record locking for read too, b'se some other thread may update if I do'nt lock the record.

    Originally posted by Andrew Monkhouse:
    The call to notifyAll() should be unnecessary.


    I understand it now, Andrew. I will try to explain the reason why I used notifyAll() before. Here is partial code for lock().

    And this is partial read code in adapter

    What I thought when I called notifyAll():
    Continuing from your example, I thought if t1 locks record 100, then t2, t3 tries to enter synchronized block. Say suppose t2 enters the synchronized block, it sees that record 100 is already locked and t1 is still processing i.e. reading record using read(). So t2 waits by calling wait() method. Now t3 enters synchronized block and locks record 200 and do notifies using notifyAll() and then continues its operations. I thought that t2 will get a chance to check again.
    After your detailed explanation, here's what I understood:
    The notifyAll() call in lock() method is unnecessary on following basis. The call to notifyAll() by t3 in lock() will make thread t2 come out of waiting state. t2 again checks for the record 100 is locked or not. Now, the thing is, record 100 will be locked as long as t1 works and before t1 calls unlock(). So till t1 calls unlock(), there is no point in t2 coming out of wait state, hence notifyAll() is not necessary and on the other hand it might be a potential cause for the 44/80 marks. So notifyAll() is MUST in unlock and not in lock().
    Andrew, I have to say this - Thanks a million!! I really appreciate your detailed explanation of things. Thanks

    Originally posted by Andrew Monkhouse:
    Personally I think that this is going to be slow and reduce concurrency, with little benefit


    That's right Andrew. I thought of it and you're right. Though I lock the database and read the record numbers and return to the client, once the find method has returned, other threads might go and change/delete the record. So, there's no point in locking the database.

    Originally posted by Andrew Monkhouse:
    You have said you are returning records, not record numbers.


    Sorry. I meant I'm returning record numbers.

    Originally posted by Andrew Monkhouse:
    Note that I did say that you could keep with your current design, as long as you document it.
    My personal preference is to try and make a more robust solution than is necessary (and normally Mark jumps in with a comment that simple designs are preferable to complex solutions).


    OK Andrew. I understand it now.

    Regards, Andrew
    Andrew, thanks. Do you think the locking is OK now? I mean if you say so also, I do not stop testing it but it will be a big positive sign for me that I am near completion of locking process, atleast
    Thanks.
     
    Satish Avadhanam
    Ranch Hand
    Posts: 697
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by Andrew Monkhouse:

    Note that returning dirty data is acceptable (and almost impossible to avoid without sacrificing concurrency). You just have to verify the data when it comes time to book the record. (You have to decide what level of checking you are going to do when you do the booking, and justify that in your design decisions document.)
    Regards, Andrew


    Andrew, one more question. You said that returning dirty data is acceptable. I am a bit worried about the locking scores coming these days. And I think all we can do to avoid is to read the correct data by locking db and once we read there is no way to keep up with other threads and updating the returned records to the customer right?
    The returned records are good only before the read/find method returns. For our project scope, once the read/find method returns, its not required to update the client side dataset. Am I right?
    Thanks Andrew.
     
    Satish Avadhanam
    Ranch Hand
    Posts: 697
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by Satish Avadhanam:

    Sorry. I meant I'm returning record numbers.


    Andrew, I thought and returning record numbers to CSR does'nt make any sense. I mean if they search for "Hilton", then returning some numbers like 130, 250, 489 does'nt make sense to CSR. So I will return the records so the CSR will know the details of the room to book.
    Thanks.
     
    Andrew Monkhouse
    author and jackaroo
    Posts: 12200
    280
    Mac IntelliJ IDE Firefox Browser Oracle C++ Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Satish,

    Do you think the locking is OK now?


    Hmmm - you have said that you understand that the notifyAll() is unnecessary in the lock() method, but you have not said that you are removing it. So I do not know whether your locking is OK or not

    I think all we can do to avoid is to read the correct data by locking db and once we read there is no way to keep up with other threads and updating the returned records to the customer right?
    The returned records are good only before the read/find method returns. For our project scope, once the read/find method returns, its not required to update the client side dataset. Am I right?


    I believe that there is no need to update the returned records.
    Regards, Andrew
     
    Satish Avadhanam
    Ranch Hand
    Posts: 697
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Andrew,

    Originally posted by Andrew Monkhouse:

    Hi Satish,
    Hmmm - you have said that you understand that the notifyAll() is unnecessary in the lock() method, but you have not said that you are removing it. So I do not know whether your locking is OK or not

    Andrew, I'm going to remove it.

    I believe that there is no need to update the returned records.

    OK Andrew.

    Regards, Andrew


    As I was going through Should lock methods be callable by the client thread, many concepts are cleared and I got basic questions on some which I thought I understand them better
    If I got any questions or I could not understand any, I will try to post them in a seperate thread so that it will be easy to discuss. That's really a very very big thread and that was a great discussion. I am also going through Ken's design and see some design choices match to him, but some are way off. I am reconsidering those and re-designing. Anyway I will post my design choices and I hope and wish you, george, phil and other big people make some comment on my design choices.
    Thanks Andrew.
     
    Andrew Monkhouse
    author and jackaroo
    Posts: 12200
    280
    Mac IntelliJ IDE Firefox Browser Oracle C++ Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Satish,

    Originally posted by Satish Avadhanam:
    I hope and wish you, george, phil and other big people make some comment ...


    Did my girlfriend put you up to that?
    OK, I am going on a diet!

    Regards, Andrew
     
    Never trust an airline that limits their passengers to one carry on iguana. Put this tiny ad in your shoe:
    New web page for Paul's Rocket Mass Heaters movies
    https://coderanch.com/t/785239/web-page-Paul-Rocket-Mass
    reply
      Bookmark Topic Watch Topic
    • New Topic