nx:All of URLy Bird 1.1.3 read/write lock and create()

[liqun chang]

Hi George:
1: your codes:

int create(String[] data) throws DuplicateKeyException {
// look for a deleted record
// when you find a candidate deleted record then
synchronized (raf)
{
// if record is deleted then write the new record here
// if record is not deleted then you'll have to keep looking
// for a deleted record
}
// if you have overwritten a deleted record then you're done
// if you haven't overwritten a deleted record then you have to
// append at the end of the file
synchronized (raf)
{
// add the new record to the end of the file
}
}

2: your mention as below:

I think this takes care of the simultaneous overwrite problem. Anytime the database file is accessed the code is placed in a block synchronized on the raf.

From your mention(2),i think about that you are right.I don't know whether
i understank your suggestion.My opinion is : In my design,i create new Data
instance for each network client,so in my create() method, i must synchronize all of actual database accessing on raf static variabe.so do i can guaranteer that has not corrupt state of database file.
In your design,because you has only one Data instance for all clients(also
singleton Data is shared by all of remote object),so you use synchronize method instead of synchronize block on raf.Am i right?

If i am right. then go to your codes of bold fonts,this process also has
actual datafile accessing,Whehter it also should be put into synchronized block?
Plese you give me some comments about this?Now i am thinking about your design(use Data singleton),and i think this scheme is more simple than me.
[ February 18, 2004: Message edited by: liqun chang ]

[George Marinkovich]

Hi Liqun,

From your mention(2),i think about that you are right.I don't know whether
i understank your suggestion.My opinion is : In my design,i create new Data
instance for each network client,so in my create() method, i must synchronize all of actual database accessing on raf static variabe.so do i can guaranteer that has not corrupt state of database file.
In your design,because you has only one Data instance for all clients(also
singleton Data is shared by all of remote object),so you use synchronize method instead of synchronize block on raf.Am i right?

Yes, you're right.

If i am right. then go to your codes of bold fonts,this process also has
actual datafile accessing,Whehter it also should be put into synchronized block?

Yes, right again.

Plese you give me some comments about this?

You might want to put the whole thing in a block synchronized on the raf.

int create(String[] data) throws DuplicateKeyException { 
    synchronized (raf) {
        // look for a deleted record 
        // if record is deleted then write the new record here 
        // if record is not deleted then you'll have to keep looking 
        // for a deleted record 
        // if you have overwritten a deleted record then you're done 
        // if you haven't overwritten a deleted record then you have to 
        // append the new record at the end of the file 
    }

So, I think the above protects the raf from simultaneous access by different threads. It doesn't provide the highest level of concurrency possible, but it does protect the raf from simultaneous access. Protecting the raf from simultaneous access is absolutely required. A solution that provides a higher level of concurrent access is better than a solution that provides a lower level of concrrent access, all other things being equal. But all other things are rarely equal. Providing a higher level of concurrent access may result in a more complicated design than one having a lower level of concurrent access. My design had a pretty low level of concurrent access. If the assignment instructions had mentioned anything about a required level of concurrent access I would have taken it into consideration, but I wasn't able to find anything on this subject in the assignment instructions. Therefore I concluded there wasn't a performance requirement. In fact, I think there is more support in the assignment instructions for simplicity and ease of understanding the solution as more desirable design goals than performance. I'll end with a quote from my assignment instructions:

A clear design, such as will be readily understood by junior programmers, will be preferred to a complex one, even if the complex one is a little more efficient.

Now i am thinking about your design(use Data singleton),and i think this scheme is more simple than me.

My design may be simpler, but your design may perform more efficiently. Either design can be well implemented. The choice is yours.

[liqun chang]

Hi George,thank you very much.Along with your suggestion,i am clear more and more.before i finish create() method,i must understand all of question on read/write lock(or be called file lock) and record lock.
you said:

You might want to put the whole thing in a block synchronized on the raf.

My purpose is only put actual file accessing code in synchronized block on
raf static variable.Certainly,some codes that don't relate to actual file
accessing should not be placed in synchronized block. Am i right?

[liqun chang]

Hi George:
from instruction:

Locking
Your server must be capable of handling multiple concurrent requests, and as part of this capability, must provide locking functionality as specified in the interface provided above. You may assume that at any moment, at most one program is accessing the database file; therefore your locking system only needs to be concerned with multiple concurrent clients of your server. Any attempt to lock a resource that is already locked should cause the current thread to give up the CPU, consuming no CPU cycles until the desired resource becomes available.

I want to sum up the core thinking about file lock and record lock.because i
think if i catch the core that you suggest,then i can implement it easily.
The primary purpose of file lock is:at the same time ,only one program is
accessing the data file(as the mention of bold fonts).so do can guarantee
maintaining the valid datafile state.Am i right?
The purpose of record lock is:if a alive client(also a thread) is locking a record(possible at the same time many threads lock many different records),
other client(thread) cann't lock this locked record at the same time.This situation guarantee each client is dealing with itself record.
So for read(),delete() and update(),we can use record lock on these methods
for exclusive reading/writting,For create() and find(),we should not use record lock on these methods for concurrent reading/writting.Am i right?

[ February 18, 2004: Message edited by: liqun chang ]
[ February 21, 2004: Message edited by: liqun chang ]

[liqun chang]

Hi George: i will go to create() method:
from instruction:

// Creates a new record in the database (possibly reusing a
// deleted entry). Inserts the given data, and returns the record
// number of the new record.
public int create(String[] data) throws DuplicateKeyException;  

my implementation of create within Data class as below:

public int create(String[] data) 
{
    int counter=0;
    synchronized(raf)
    {
        seek(start of record data);
        do
        {
            read();//read flag
            raf.skipBytes(recordLength);
            counter++;
            //check deleted flag
            if(isDeletedFlag)
            {
                break;
            }
        }
        while(raf.read()!=-1);
        seek(raf.getFilePointer()-(recordLength+1));
        raf.write(validFlag);
        raf.write(recordData);
    }
    return counter;
}

1: In create method,i don't declare DuplicateKeyException,because the specification is not clear,i want document it to doc.
I will wrap IOException and other exception into DataAccessException(a subclass of RuntimeException).Am i right?
2: Because the codes of actual accessing data file are too much,i decide to put most of codes in synchronized block on raf.
perhaps this has lower performance.Please you help me? whether has other skill?
3: The most important is: whether my logic algorithm is right? Sorry for so material question,because you are a good teacher. I like discuss with you and very trust you.
4: About RandomAccessFile,i will look at API doc,if any question,i will ask you.
Hi Philippe and Andrew,could you give me some suggestions ?
[ February 19, 2004: Message edited by: liqun chang ]
[ February 19, 2004: Message edited by: liqun chang ]
[Andrew: put code inside [code] blocks]
[ February 19, 2004: Message edited by: Andrew Monkhouse ]

[Andrew Monkhouse]

Hi Liqun,
As you can see, I edited your message to put the code inside [code] blocks. If you edit your message you will be able to see how I did this. [code] blocks are preferable to [quote] blocks when displaying code, as the [code] block will maintain indentation for you, which makes the code easier to read.
I think you will have a problem if there are no deleted records. Instead of adding to the end of the file, you will overwrite the last record. And having overwritten record 'x'-1, you will return 'x' to the calling method - a number which does not exist.
Regards, Andrew

[liqun chang]

Hi Andrew:you are right,i will modify my codes as below:

public int create(String[] data)
{
    int counter=0;
    int flag=0;
    synchronized(raf)
    {
        seek(start of record data);
        while(true)
        {
            flag=raf.read();//read flag(i have already correct it) :-P	 
            raf.skipBytes(recordLength);
            counter++;
            if(flag is deleted || flag==-1)
            {
                seek(raf.getFilePointer()-(recordLength+1));
                raf.write(deletedFlag);
                raf.write(data);
                break;
            }
        }
        return counter;
    }
}

Am i right for create method?please you give me some comment?Another question is whether when at the end of file,any write() or skipByte() will
not move the file pointer?because in API doc,i could not find any discussion
about this point.Could you give me some detail about the file pointer?
[ February 20, 2004: Message edited by: liqun chang ]
[Andrew: indented code]
[ February 20, 2004: Message edited by: Andrew Monkhouse ]
[ February 20, 2004: Message edited by: liqun chang ]
[ February 20, 2004: Message edited by: liqun chang ]
[ February 22, 2004: Message edited by: liqun chang ]

[liqun chang]

Hi George,where did you come to?i am waiting for you.please you help me about the file lock and record lock above?
Hi Andrew,i do code in code block,but it is also in that way without indentation.

[Andrew Monkhouse]

Hi Liqun,
Getting closer, but still not quite there yet.
General typo - you have the code "flag = raf.write();//read flag". I think you mean to read data, not write it.
If you have reached the end of the file (indicated by having read "-1"), you have still skipped the size of a record, so you will now have a gap between the previous "last record" and the record you just wrote.

Another question is whether when at the end of file,any write() or skipByte() will not move the file pointer?because in API doc,i could not find any discussion about this point.Could you give me some detail about the file pointer?

Yes, that is mentioned in the JavaDoc ("The offset may be set beyond the end of the file. Setting the offset beyond the end of the file does not change the file length. The file length will change only by writing after the offset has been set beyond the end of the file."). There is a similar comment about writing.
Regards, Andrew

[liqun chang]

Hi Andrew,i will modify my code again from original location.pelease you see if i understand what is your suggestion?
For bold fonts above please you give me some comments about it?

[liqun chang]

Hi George and Andrew:
instruction 1:

// Returns an array of record numbers that match the specified
// criteria. Field n in the database file is described by
// criteria[n]. A null value in criteria[n] matches any field
// value. A non-null value in criteria[n] matches any field
// value that begins with criteria[n]. (For example, "Fred"
// matches "Fred" or "Freddy".)
public int[] find(String[] criteria);

instruction 2:

It must allow the user to search the data for all records, or for records where the name and/or location fields exactly match values specified by the user.
It must present search results in a JTable.

My questions are:
1: From the instruction 2,whether i will present all of field values of a record as a row in JTable,also as follow:

name location size smoking rate date owner
name location size smoking rate date owner
name location size smoking rate date owner
Am i right for this situation?
2: From the instruction 1,the returned value is the int[] which represents the record numbers.In fact the find will be used
in the serarch() method,but the search() method is required providing more function to return search result.Am i right ?
and whether i understand the essential of the instruction?
3: please you give me the general suggestion about the find() and search() methods?
Hi George your comment:

Your understanding of the logic of requirement is technically correct. 1 and 2 are relatively easy to implement. Implementing
3 is difficult to do in a single search. Many people (maybe most people) are not supporting 3 in a single search. Instead
they are supporting 3 as two independent searches.
3a: records where the name field exactly matches values:
(criteria[] name null null null null null), and
3b: records where the location field exactly matches values:
(criteria[] null, location, null, null, null, null)

4: I will create two JTextFields for inputing the name and location for serach() method,and a button with "search" ,If user only inputs in name field and click button,then searches for records that only name matches,if user only inputs in location field and click button,the searches for records that only location matches,if user inputs in name and location fields then searches for records that name and location match,if leave two fields empty then searches all records.Am i right?I will use case sensitive for search,am i right?
[ February 21, 2004: Message edited by: liqun chang ]

[Andrew Monkhouse]

Hi Liqun,

Hi Andrew,i will modify my code again from original location.pelease you see if i understand what is your suggestion?
For bold fonts above please you give me some comments about it?

This looks quite good now.
The only remaining problem is that section which I had a comment in bold for before - where you are writing to the disk, but your comment next to the write statement says that you are doing a read.

When you compile and test this code, you will see what I mean (make sure you have a backup of your original database!)
Regards, Andrew

[Andrew Monkhouse]

Hi Liqun,
Your questions on the search methods are not related to the subject of this thread ("nx:All of URLy Bird 1.1.3 read/write lock and create()") - you really should consider opening a new thread when you change topics. People who were not interested in reading about the read/write lock may be interested in the search methods. And if someone wants to post a follow up message about the read/write lock it may be very confusing to see the response in the middle of a discussion about searching.

Hi George and Andrew:

Also it is a good idea not to post questions directly to individuals like this - other people may decide not to answer since the question is addressed to George and myself.
As for your questions:

Yes

Yes

Sorry, it doesnt work like that. You need to develop the submission yourself. There is no value in me providing potential solutions to you - you will not have gone through the learning exercise yourself.
If you would like to suggest some ideas for any topic and get feedback on whether it is workable or not, that is OK.

This sounds very good.
Some people have also considered using pull down lists which contain the possible values of the database. This makes it easier for the end user, but at the expense of requiring more network traffic and becoming more unwieldy as the database grows.

Regards, Andrew

[George Marinkovich]

Hi Liqun,

Originally posted by liqun chang:
My purpose is only put actual file accessing code in synchronized block on
raf static variable.Certainly,some codes that don't relate to actual file
accessing should not be placed in synchronized block. Am i right?

Yes you're right. In general you want the size of the synchronized block to be small as possible.

[George Marinkovich]

Hi Liqun,

Originally posted by liqun chang:
The primary purpose of file lock is:at the same time ,only one program is
accessing the data file(as the mention of bold fonts).so do can guarantee
maintaining the valid datafile state.Am i right?
Yes.
The purpose of record lock is:if a alive client(also a thread) is locking a record(possible at the same time many threads lock many different records),
other client(thread) cann't lock this locked record at the same time.This situation guarantee each client is dealing with itself record.
So for read(),delete() and update(),we can use record lock on these methods
for exclusive reading/writting,For create() and find(),we should not use record lock on these methods for concurrent reading/writting.Am i right?Yes

[ February 21, 2004: Message edited by: George Marinkovich ]

[George Marinkovich]

Hi Liqun,

Originally posted by liqun chang:
1: In create method,i don't declare DuplicateKeyException,because the specification is not clear,i want document it to doc.
I will wrap IOException and other exception into DataAccessException(a subclass of RuntimeException).Am i right?
Yes, given your assignment (URLyBird) that seems to be the best way to handle the DuplicateKeyException. Yes.
2: Because the codes of actual accessing data file are too much,i decide to put most of codes in synchronized block on raf.
perhaps this has lower performance.Please you help me? whether has other skill?
Seems reasonable to me.

3: The most important is: whether my logic algorithm is right? Sorry for so material question,because you are a good teacher. I like discuss with you and very trust you.
Your algorithm generally looks OK to me. But you shouldn't rely just on reading the code. I hope you're planning on unit testing the Data class. Especially you should test the boundary conditions. What happens: when you create a record in a data file that contains no other records, when you create a record in a data file that only contains deleted records, when you create a record in a data file that doesn't contain any deleted records? Try creating a record in a file without any deleted records. Then create another record in that file. Try a data file without any deleted records, delete a record, then create a record. Does the created record end up where you would expect? The point is I'm not going to be as good at catching errors reading the code as unit testing will be.

[liqun chang]

Hi Andrew you say:

The only remaining problem is that section which I had a comment in bold for before - where you are writing to the disk, but your comment next to the write statement says that you are doing a read.

OK,it is my mistake.thanks for you.I will correct it.
and you say:

Sorry, it doesnt work like that. You need to develop the submission yourself. There is no value in me providing potential solutions to you - you will not have gone through the learning exercise yourself.
If you would like to suggest some ideas for any topic and get feedback on whether it is workable or not, that is OK.

Thanks for you again.only through myself work,then i have the experience.
I will according to your suggestion do it myself.If have any questions,
i will remit to you.
you say:

Also it is a good idea not to post questions directly to individuals like this - other people may decide not to answer since the question is addressed to George and myself.

I should not according you suggestion about this,haw-haw because i know
you from this forum,you are a high level expert.I like your style:simple and
clear answer.i have already read a lot of articles that come from you and get a lot of skill about java(not only scjd).
About George,he is the first friend from i come to this forum.He give me a
lot of help at the time that i am being most difficult.
[ February 22, 2004: Message edited by: liqun chang ]

[liqun chang]

Hi George,thanks for your reply.you are the expert that always gives me help.please see the end of previous post.
1: For creating across platform applications,i am always use the JCreator or text editor.My question is whether there is a accross platform IDE for java.Please you give some suggestion?
2: Anoter question is:i see someone discuss about Ant and JUnit,what use for them?Could you give me some simple suggestion?
3: About the JUnit,whether you could recommend a simple guide handbook for me (the best is the ebook)?
According to Andrew's suggestion,i will create a new thread and ask you about find and search method.

nx:All of URLy Bird 1.1.3 about find and search method
please you give me some suggestions on that thread.OK?

[George Marinkovich]

Hi Liqun,

    UIManager.setLookAndFeel
        (UIManager.getCrossPlatformLookAndFeelClassName());

Avoid hard-coding paths, if you do don't rely on "\" path separators, but use the System.getProperty(path.separator) (that's not it exactly but it's close).
2: Anoter question is:i see someone discuss about Ant and JUnit,what use for them?Could you give me some simple suggestion?

Ant is used to build your application and JUnit is used to test your application. You don't need either for this project, but you do need to build your application and you absolutely need to unit test your application (especially Data, since there are several methods in the class that aren't used elsewhere in you application: create and delete for example). Both of these things are nice to know but they aren't required for the project. You can get ant from http://jakarta.apache.org/ant.
3: About the JUnit,whether you could recommend a simple guide handbook for me (the best is the ebook)?
You might check out their web site: http://www.junit.org/ They probably have book recommendations there.
[ February 24, 2004: Message edited by: George Marinkovich ]