nx:All of URLy Bird 1.1.3 about read/write lock

[liqun chang]

Hi Andrew,lovely guy,i always don't know why you don't answer my questions
I understand now.Although i always study the java for about six years,because in China,i can't find high-level experts to discussion.Today
,when i receive your reply,i call my wift(a chemistry teacher)and see you
reply all together.Because i am doing scjd,please you and other experts help
me.Because i am a teacher,i am spreading abroad to my students,hope you become the bridge between java technology and chinese students.

1: From you reply,i guess if i remove notifyAll() from my lock method,i am ok.Am i right? Say really,i think that multithread is important technology in java,then i use about six months to see this section.About communicatin
between threads,i also don't think notifyAll() is necessary within lock method,only because Max's suggesting in anoter thread,because the article is long and there is a lot of discussion on it,i didn't finish reading.
2: I am always thinking about Max'sugesting,if i use the only one mutex that either for record lock or file read/write lock,is the notifyAll() method that is located in lock method is necessary?
Could you give me some suggest?
In my design(as mention in the next section),I decide to use two mutexes
(WeakHashMap object and RandomAccessFile object).Am i right? or i use
FileChannel instead of RandomAccessFile.because i know FileChannel a little
,i hope to use it to get some experience.
Please you give me reply,and George where did you come to?
Ok wait for your reply,and then i will go to read/write lock seciton.
[ February 12, 2004: Message edited by: liqun chang ]
[ February 12, 2004: Message edited by: liqun chang ]

[liqun chang]

The purpose which i do scjd is to learn the core java technology.So before i mention the read/write lock.I will discussion
about thead safe.Hope you give me some comments.
1: In a class or object,variable is private and method public can't guarantee thread safe.Multiple thread is built in java
and API,all threads inside JVM share the same heap and method area.You need only be concerned with instance variable and class variable when you worry about thread safe. Heap is the memory location where all instances variables are stored. Method area
is the location where all class variables(or be called static variable) are stored.Don't worry about local variables,method
parameters and return value,because they reside on Stack.Multiple threads attempt to use the same object's instance variable
and class variable concurrently.
Is this sentence right? If have mistakes,please give me some cmomments on the detail section.

2: If you design the class that for concurrent access,then you must think about the instance variables and staitc variables,
also you must consider the methods that read from/write to two kind of variables(instance or static) .So do,you can maintain
the valid state without corrupting fields.
Am i right? If not please you give me a comment.

[George Marinkovich]

Hi Liqun,

Originally posted by liqun chang:
1: From you reply,i guess if i remove notifyAll() from my lock method,i am ok.Am i right? About communicatin
between threads,i also don't think notifyAll() is necessary within lock method,only because Max's suggesting in anoter thread,because the article is long and there is a lot of discussion on it,i didn't finish reading.
Yes, I would recommend removing the notifyAll() from the lock method, because if you think about it, no thread really cares to be told when a record has been locked. Threads do care about being told when a record is no longer locked, because a thread may be waiting (in the lock method) for that record to become unlocked.
2: I am always thinking about Max'sugesting,if i use the only one mutex that either for record lock or file read/write lock,is the notifyAll() method that is located in lock method is necessary?
Could you give me some suggest?Same comment as for 1 above. I think you can safely remove the notifyAll from the lock method. When the end of the synchronized block of code is reached, all other threads that may be waiting for that synchronized block of code will be notified. That is sufficient, you do not need to also notify threads waiting for your locking variable, since adding a record to the locking variable can never help a thread waiting for a lock to become available. The unlock method, in which a record is removed from the locking variable, is of interest to any thread waiting in the lock method, and hence that is why the notifyAll is present in the unlock method.
In my design(as mention in the next section),I decide to use two mutexes
(WeakHashMap object and RandomAccessFile object).Am i right? or i use
FileChannel instead of RandomAccessFile.because i know FileChannel a little
,i hope to use it to get some experience.
I don't recommend the use of a FileChannel for this project as it appears to be explicitly prohibited by a document posted on the Sun site (you can see this discussed here: Topic: NIO prohibited for SCJD project?.
Please you give me reply,and George where did you come to?
Ok wait for your reply,and then i will go to read/write lock seciton.

It's probably human nature (at least it's my nature) to answer shorter, simple questions first. Longer questions or questions that require me to analyze something ("is this design OK? or is this code OK?"), require some time for thought, and therefore take me longer to respond. I hope you understand.
Hope this helps,
George
[ February 12, 2004: Message edited by: George Marinkovich ]

[George Marinkovich]

Hi Liqun,

Originally posted by liqun chang:
I will discussion
about thead safe.Hope you give me some comments.

I agree with what you've written. I don't think you have mis-stated anything.
Hope this helps,
George

[liqun chang]

Hi George:
as you say:

I don't recommend the use of a FileChannel for this project as it appears to be explicitly prohibited by a document posted on the Sun site (you can see this discussed here:

I see that thread and see also Sun's website.I decide do not use the
FileChannel,and i will use RandomAccessFile directly.Your mention is right,
we must use the clear specification by Sun without any confusion.

[Jamy Wang]

Hi liqun, glad to meet you here. I'm also from China. Just like you, I'm taking SCJD exam too.
I'm still a little confused about the lock mechnism. Please forward this thread: https://coderanch.com/t/185126/java-developer-SCJD/certification/NX-read-write-lock-mechnism
If you like, we can communicate with each other about lock mechnism and other points in the exam. My QQ:43116737. Email: [e-mail]

[liqun chang]

Hi james,glad to meet you too.I have seen your email and qq number.I hope we
become friend because you are the first chinese scjp that i meet.I hope we discussion all of the exam in this forum because the written exam.In forum we can gain the help of Andrew,George and other experts,and can exercise
logic and clear answer and analyse the problems.After finish the scjd(certainly we all pass).We talk about through qq,did you agree me?and i belive we can become friend.

[liqun chang]

Hi George,thanks very much and i understand that for longer problem,you must be thinking about for long time.I am a little impatient.
As you said:

I have a concern about the pseudocode in the isValidated method, however. If I understand correctly, isValidated is called whenever lock is called (in other words, it's called more than one time). The pseudocode indicates that the header information is read from the database file and then all the records are read. This is a lot of work to do each time the isValidated method is called (which is each time the lock method is called). My suggestion is that the header information in the database file should be read only once (perhaps when the Data class is instantiated). The isValidated need only seek to the correct location for the recNo and read the validity flag to determine: 1) whether the record physically exists in the database file, and 2) whether the record has been marked for deletion or not.

According to your suggestion,I will create DataSchema singleton class ,and all of the metadata is stored in this class.
When construct Data, read all of metadata in DataSchema. Is this all right?
The DataSchema instance is shared by all of Data instances(also by all of the network clients).

But in this situation the questions will raise.
1: my isValidated(recNo) method

public void isValidated(recNo) throws RecordNotFountException
{
//raf is another mutex
synchronized(raf)
{
raf.seek(position);
}
//read record that specified by recNo
//judge whether record is really existing in datafile,if not throw RecordNotFoundException,else go on
//judge whether record has deleted flag,if yes throw RecordNotFoundException,else go on
}

In this situation,there three possible problems. a:synchronized on seek. b:synchronied on write/read. c:synchronized on
seek and write/read.Which is correct? please you tell me and give some comments.

2: Because i want use two mutexes(the staitc WeahHashMap and the static RandomAccessFile),in lock and other methods,
for example in lock():

lock
{
isValidated(recNo);
synchronized(lockMap)
{
//some codes.
}
}

Because i specify an order of synchronyzed block,so i shoud not worry about the problem of deadlock.Because there is not
the situation that a synchronized block nest another syncronized block,so there is not deadlock issue.
Am i right? If have mistakes,please you give me comments on detail section.

3: If i want use only one mutex that for record lock and file read/write lock,then i will use the local RandomAccessFile
variable for read/write.
the isValidated(recNo) look as below:

public void isValidated(recNo) throws RecordNotFoundException
{
RandomAccessFile raf;
raf=new RandomAccessFile(file);
synchronized(lockMap)
{
//while recNo in lockMap
lockMap.wait();
//else read record and judge recNo validity.
}
}

Although i didn't use this situation,but i want to know whether this method is right.
4: Frome other threads,i hear about concurrently read and exclusively write,exclusively read and exclusively write.
I really don't know what is the mean.Could you help me?and give me a simple example,thanks.

5: And as Max said:"the key is write and write conflict".Personal point of view is read and write must be synchronized on
the monitor because if a reading is going and at the same time a writting is going on temporary state,the result will be
wrong. Am i right? please you give me comments.or Max'suggesting is below a special context,i really don't understand?

6: Perhaps this topic shoud be located before 3,i will create Record class as the oo framework that wraps the record data
byte array.And i also create DataInfo class for storeing Record with appropriate Collection.So do,i can easy use MVC model for JTable display.This is my inital idea.
Am i right? Could you give me some detail suggestion?

[George Marinkovich]

Hi Liqun,
I think the DataSchema class is the way to go.
1. I think the correct thing to do is to put the seek, read, and write, all within the same block synchronized on raf. This is your answer c. The reason is this: you want the seek and read (or seek and write) to be atomic. In other words, you seek somewhere in the file and then you want to read at that location. But you want these two actions to occur as if they were a single action. For example, you don't want thread A to seek somewhere, then thread B to seek somewhere else (thus moving the file pointer), and then thread A to read (not from where thread A seeked to, but instead) from where thread B seeked to. You just want thread A to do its business (seek and read) in peace without interference from thread B. Putting the seek and read in the same block of code synchronized on raf accomplishes this (making the seek and read atomic).
2. I think you're OK here because as you say the synchronized blocks are not nested. Another way to say this is that you don't have to acquire both of your locks in order to do something. You acquire one lock (in the isValidated), then give this lock up, then acquire another lock (on lockMap), then give this later lock up. I don't see any potential problems here since at any point in time you only ever hold one lock.
3. I don't like the isValidated implementation here. I don't like opening the raf every time this method is called. Think about what happens when you get all the records from the database. First you call find which examines each record in the database (for which isValidated is called), so the raf is being opened once for each record in the database. Then when you get the array of recNos from find you call read for each record in the database (for which isValidated is called), so the raf is being opened once for each record in the database (again). So, for a find that gets all the record in the database, you are calling isValidated, (which is opening the raf) twice as many times as there are records in the database. I think this is definitely a worse implementation.
4. When you update the database I think you always want to do it exclusively. That is, when you modify the database you always want to do that exclusively. For example, when you call update(recNo), you always want to do: lock(recNo), update(recNo), unlock(recNo).
When you read the database you can decide whether you want to do this exclusively or not. For example, you may decide that when you call find() you want to get the records on a non-exclusive basis. That is, you don't want to have to lock every single record in the database before you execute the find. Because your find is executing without respect to locks it's possible that you get some records for which another thread has a lock. So you can potentially get records that are about to be changed by another user (or even in the process of being changed). You might decide this is an acceptable tradeoff: in exchange for not bothering about locks you get to read everything quickly (without waiting for locks), but some of what you have read might be about to change (for instance, because someone else has locked this record before doing an update).
There may be a circumstance in which you want to read something exclusively (like right before you're about to update the record). So if you think about the booking operation you probably want to do something like: lock(recNo), read(recNo), update(recNo), unlock(recNo). Here the read is happening exclusively (because it occurs after a lock and before an unlock).
So the short answer to your question is: You always want to write exclusively. Sometimes you want to read exclusively (when it's very important that you get the latest values for a record), and sometimes you do not need to read exclusively (when you don't really care that you're getting the absolutely latest values for a record).
5. I'm not sure if I fully understand Max's point here ("the key is write and write conflict"). Should it be: "the key is read and write conflict?" I guess I agree, but it comes down to the situation. When I'm about to update a record I want to get it's latest values, and I'm willing to pay the price to lock(recNo), read(recNo), and unlock(recNo). When I'm just executing a user's search request, I'm willing to take my chances and just read what's there without doing the locking. Doesn't that mean that sometimes the user may be looking at something out-of-date (or even worse something partially written)? Yes, that's true, but it's not really harmful because if the user decides to book a stale record, as part of that booking operation, I will do an exclusive read (thereby picking up the latest values for that record).
6. You can create a Record class that will encapsulate the record data. Or you could just use String[]. It's up to you. You can also create DataInfo class as you say, or you might find that when you start implementing your table model that you can just handle the collection there as a member variable in the table model class. There's nothing wrong from an OO perspective with the classes you propose, it's just that you may decide as you get further involved in the implementation that you don't get a lot of additional benefit by making them their own class. This is something that will become more clear as you proceed.
Hope this helps,
George

[liqun chang]

Hi George,thanks for your reply.i am clear bit by bit.
You say as follow:

I think the DataSchema class is the way to go.

1: According to your suggestion,I create DataSchema that contains metadata.and in next section to mention it.

2: About the isValidated() method:
you say:

1. I think the correct thing to do is to put the seek, read, and write, all within the same block synchronized on raf. This is your answer c. The reason is this: you want the seek and read (or seek and write) to be atomic. In other words, you seek somewhere in the file and then you want to read at that location. But you want these two actions to occur as if they were a single action.

According to you suggestion,i will changed my code of isValidated as follow

public void isValidated(recNo) throws RecordNotFoundException
{
//raf is new mutex
synchronized(raf)
{
raf.seek(position);
raf.read();
}
//judge whether recNo is really exist in datafile,if not throw RecordNotFoundExcption,if not go on
//judge whether recNo has deleted flag,if has throw RecordNotFoundException,else go on
}

Am i right for changed code?
About atomic:If methoda and methodb be located in one method(or be called in one action),then we can say that we make methoda
and methodb atomic.Then methoda and methodb must run finished at the one time.
Am i right?
3: For your suggestion:

3. I don't like the isValidated implementation here. I don't like opening the raf every time this method is called. Think about what happens when you get all the records from the database. First you call find which examines each record in the database (for which isValidated is called), so the raf is being opened once for each record in the database. Then when you get the array of recNos from find you call read for each record in the database (for which isValidated is called), so the raf is being opened once for each record in the database (again). So, for a find that gets all the record in the database, you are calling isValidated, (which is opening the raf) twice as many times as there are records in the database. I think this is definitely a worse implementation.

Thanks for you suggestion.i will not use this mode.I will use two mutexes for read/write lock and record lock.Ok?

4: About concurrent reading and exclusive writting,exclusive reading and exclusive writteing.
About the exclusive writting(for example:update,delete),i need to lock the special record and do writting operation,also
use the lock() and update() and unlock() in a single action.
About the concrrunt reading(be used find),i don't need to lock the special record,so do i will get some records that perhaps one
record is locked by another thread(or is being modified by anoter therad),Also some of these records that be readed are not
the lastest record.
(1): Whether you suggest me to use read() that there is not lock before it for concurrent reading? and let find to invoke read?
(2): Whether i need to worry about the concurrent reading at the same time there is exclusive writting?Because when i have read
some record and display them in JTable at the same time(or only after reading) some records has been changed.Is this conform
to Sun specification?
(3): About the exclusive reading,i need to lock special record and do reading operation,also use the lock and read and unlock as an order.Am i right?
5: As you mention as follow:

I'm not sure if I fully understand Max's point here ("the key is write and write conflict"). Should it be: "the key is read and write conflict?" I guess I agree, but it comes down to the situation. When I'm about to update a record I want to get it's latest values, and I'm willing to pay the price to lock(recNo), read(recNo), and unlock(recNo). When I'm just executing a user's search request, I'm willing to take my chances and just read what's there without doing the locking. Doesn't that mean that sometimes the user may be looking at something out-of-date (or even worse something partially written)? Yes, that's true, but it's not really harmful because if the user decides to book a stale record, as part of that booking operation, I will do an exclusive read (thereby picking up the latest values for that record).

Oh George,Max's mention is absolutely "the key is write and write conflict",or else i shoud not get some confusion.
From this section,Whether you suggest me that if i want get the latest values of records,i should use the exclusive
reading(use the order of lock and read and unlock),also used in book().
When call the find and then invoke the read,this is the situation that use the concurrent reading,also only invoke the read
without lock before it and without unlock after it.The purpose of the action is to get the record that may not the latest
value.I do not know whether i really understand your suggestion.If yes, i will use concurrent reading in find and use exclusive read in book(lock and read and unlock) and use exclusive writting in book(lock and update and unlock).In fact,
there only two business methods search and book(lock and read and update and unlock).
Am a right? and you know this is the key about the read/write,If i has a little mistakes,please you find out? thanks very much!
6: Hi George,from you mention as below:

You can create a Record class that will encapsulate the record data. Or you could just use String[]. It's up to you. You can also create DataInfo class as you say, or you might find that when you start implementing your table model that you can just handle the collection there as a member variable in the table model class. There's nothing wrong from an OO perspective with the classes you propose, it's just that you may decide as you get further involved in the implementation that you don't get a lot of additional benefit by making them their own class. This is something that will become more clear as you proceed.

Is the bold fonts mean that "it's lied on me"? Sorry for my english.And for purpose of simple and clear reason,i think i will create class instead of variable(certainly i prefer to java core class).
Hi George waiting for you reply.and Andrew,Mark,and Bharata,could you also help me?
If i finish this section.I will go to my read and find in Data class implementation.
[ February 14, 2004: Message edited by: liqun chang ]

[Andrew Monkhouse]

Hi Liqun,

i always don't know why you don't answer my questions

As George noted - shorter, easier questions get looked at first, then if I still have time, I look at the harder or longer questions. Another big issue can simply be when you write - if I am busy at work (as I have been this last week) I may not have time to answer questions - or if I have to go to head office (as I did this week) then I have very limited access to the internet, so I cut down on how many posts I respond to.
The good thing is that there are many people (such as George and others) who are willing to provide answers to others, so you do not need to wait for any specific person to get a response.

About atomic:If methoda and methodb be located in one method(or be called in one action),then we can say that we make methoda
and methodb atomic.Then methoda and methodb must run finished at the one time.
Am i right?

Just having them in the same method will not make them atomic - the JVM scheduler can swap one thread out while it is in the middle of a method. What you need (and what you seem to be doing) is to put the code inside a synchronized block.

About the exclusive writting(for example:update,delete),i need to lock the special record and do writting operation,also
use the lock() and update() and unlock() in a single action.

You do not need to use the lock() and update() and unlock() within a single (atomic) action, nor do they necessarily all need to be in the same method (although most candidates appear to be doing this).
The lock() and unlock() method perform logical locking and unlocking of a record. You have to make portions of the update() method atomic, since you cannot allow a different thread to change an underlying object (e.g. the file pointer) before you get to do your write to file. But there is no such problem with the lock(), update(), and unlock() operations - if the three method calls are not within an atomic block, it will not matter. No other thread can change the logical lock that was granted to you, so when your thread gets sliced in again, everything will still be in the same state.

(1): Whether you suggest me to use read() that there is not lock before it for concurrent reading? and let find to invoke read?

Agreed - you should not need to have a logical lock on a record before you call read().
Note that read() itself should have a synchronized block to ensure that it's file pointer does not get changed by another thread.

(2): Whether i need to worry about the concurrent reading at the same time there is exclusive writting?Because when i have read
some record and display them in JTable at the same time(or only after reading) some records has been changed.Is this conform
to Sun specification?

I would recommend that you put a comment in your design decisions document, stating that you did consider this issue, then decided that it was beyond the scope of the assignment. (And give your reasons).
Your code will always be returning a valid record - it will never return a record that has half a field updated. The record read may be dated (it may have been modified since you read it) but it will never be in an invalid state.
So all you have to care about is what effect having dated records might have - probably it just means that when the user tries to book a record, they will get an error message stating that the record has already been booked. (This is what happens in most airplane / car / hotel booking systems currently in use in the real world).
If you did decide to guard against this, you will have to consider what level of transaction isolation you want for your database operations, and this is definately getting beyond the requirements of the assignment.
Regards, Andrew

[liqun chang]

Hi Andrew,for get you reply,i prefer to keep my question simple and clear.
Hi George,whehter you are busy too.
1: About atomic:
Hi Andrew you say:

Just having them in the same method will not make them atomic - the JVM scheduler can swap one thread out while it is in the
middle of a method. What you need (and what you seem to be doing) is to put the code inside a synchronized block.

As your mention,If i put a set of methods in a synchronized block or in a synchronized method,then i make these methods atomic.Also i make a set of methods as a single action without interrupting.For parts of code,i can put them in to synchronized block or put them in the synchroized method,this also be called to make them atomic.Am i rigth?thanks very much.

2: Hi Andrew you say:

You do not need to use the lock() and update() and unlock() within a single (atomic) action, nor do they necessarily all need to be in the same method (although most candidates appear to be doing this).

I already understand.According to you suggestion.I will put lock,update,unlock in a method(not synchronized block or
sychronized method),this is called exclusive writting other then atomic operation.Am i OK?

3: as you say:

The lock() and unlock() method perform logical locking and unlocking of a record. You have to make portions of the update() method atomic, since you cannot allow a different thread to change an underlying object (e.g. the file pointer) before you get to do your write to file.

Thanks for you very clear answer.I will make portions of the update method in synchronized block.Also put raf.seek() and raf.write() in synchronized block.(or put the physical writting code in synchronized block).Am i right?

4: you say:

But there is no such problem with the lock(), update(), and unlock() operations - if the three method calls are not within an atomic block, it will not matter. No other thread can change the logical lock that was granted to you, so when your thread gets sliced in again, everything will still be in the same state.

I guess whether you suggest me:if threada locks a record then no other thread can get the same record lock until threada unlock this record.So do,we can guarantee only one record can be modified by only one thread at the same time.Am i right?

5: your mention as follow:

Note that read() itself should have a synchronized block to ensure that it's file pointer does not get changed by another thread.

I try to use your term.I will make raf.seek() and raf.read() atomic,Also put them in synchronized block of read method.Am I right?

6: you say:

Your code will always be returning a valid record - it will never return a record that has half a field updated. The record read may be dated (it may have been modified since you read it) but it will never be in an invalid state.
Why? Whehter because i make seek() and read() atomic(certainly contains seek() and write()).Personal opinion,only synchroized mechanism can result in the valid state.Am i right?

7: Hi Andrew you say:

If you did decide to guard against this, you will have to consider what level of transaction isolation you want for your database operations, and this is definately getting beyond the requirements of the assignment.

Please you give me a clear comments about the bold fonts.please? Sorry for english is not my mother language.
My opinion is i will provide synchronized read / write,either in high level or in low level,and use the wait()/notify() for communication between threads.Am i right?
[ February 14, 2004: Message edited by: liqun chang ]

[Andrew Monkhouse]

Hi Liqun,

Yes

You should be OK with this.
But I do not agree with your statement "this is called exclusive writting other then atomic operation" - there is nothing exclusive once you have your code in a non synchronized block.

"I will make portions of the update method in synchronized block.Also put raf.seek() and raf.write() in synchronized block"
This sounds like you have two synchronized blocks. I think only one is needed.

"So do,we can guarantee only one record can be modified by only one thread at the same time".
This is a completely separate topic, and if you wanted to explore it, then you might want to open a new thread to discuss it. You do need to think about deadlock issues, and limiting each client to only being able to lock one record at any given time is one way to achieve this. Another way is to force locks to be granted in a specific order. A third way is to have a dedicated deadlock detection method/class.

Yes

If you do not have your file access routines in synchronized blocks, then you cannot guarantee that you will get any usable data at all.
If you have all calls to file access routines in synchronized blocks, then you cannot get invalid data.
This is where I am differentiating between dated data (data that was correct at the time it was read, but which may have been modified since then) and invalid data (data which has been partially updated / the wrong record / total garbage).

"Please you give me a clear comments about 'what level of transaction isolation you want for your database operations'".
Take a look at the topic "Why are dirty reads ok? - in particular the section where I start by stating "This post is just to clear up a few terms that are being used here. You do not need to know about this for SCJD.". In that I discuss isolation levels with respect to database operations.
My comment to you was meant to be a way of explaining the level of detail that we do not want to go to in this assignment. Sorry that it raised a whole new set of questions.

"My opinion is i will provide synchronized read / write,either in high level or in low level,and use the wait()/notify() for communication between threads.Am i right?"
In general, synchronized blocks will cause bottlenecks in your application, so normally you would want to have them at a low level to reduce the bottlenecks.

Whew - so many questions again
Regards, Andrew
[ February 15, 2004: Message edited by: Andrew Monkhouse ]

[liqun chang]

Hi Andrew and George,Sorry for fazing you,but i prepare to faze you forever.
But i will reduce the number of questions and more clear and simple.

I will open the new thread for read/write lock
3: nx: All of URLy Bird 1.1.3 read/write lock(2)
[ February 15, 2004: Message edited by: liqun chang ]