Passed SCJD

[A Mesbah]

Thanks to all the great developers here on Javaranch. I did not post any questions but I read the posts of others constantly.


The maximum possible score is 400; the minimum to pass is 320.
General Considerations (maximum = 100): 86
Documentation (maximum = 70): 70
O-O Design (maximum = 30): 30
GUI (maximum = 40): 31
Locking (maximum = 80): 44
Data store (maximum = 40): 40
Network server (maximum = 40): 40

Total: 341

The 44 bug on Locking! That was my greatest concern.

Thaks again and happy new year
[ January 03, 2005: Message edited by: A Mesbah ]

[David Abramowicz]

congratulations!

[Sean Gildea]

congrats!

how many hours did you spend on the project altogether you think?

[Jim Janssens]

And what was the bug in your locking system ?

[A Mesbah]

I spent about two whole solid weeks on the assignment. I did a lot of reading (Javaranch) before starting the implementation.

I used a test-driven approach using Maven, Junit and Clover. The Java Logging API was used extensively through out the whole project.

The locking bug is a mystery to me because I had a whole lot of junit-tests for the locking part!!!

[Steven z.Li]

congratulations!I just in preparing SCJD,will you share your experience?

[A Mesbah]

Assignment: Bodgitt and Scarper

- I have a 3-tier design with a very simple application life-cycle;
The three tiers are:
1. Data access layer
2. Services layer (which can be local or networking)
3. Client layer

- For networking I have chosen RMI.

- The Data class implements the DB interface specified by Sun.
Data class delegates all its data access to a helper class
called DataHelper. All of the synchronized code is in DataHelper which
is implemented as a singleton.
DataHelper uses a static RandomAccessFile directly
to get to the data file. When initialized, DataHelper (using
RandomAccessFile) checks to see if the data file exists, it validates the
data file against the magic number, reads the fields number and the
schema. All the methods
in (DataHelper class) that use the RandomAccessFile are synchronized to
avoid manipulating of the pointer of the RandomAccessFile by different
method calls. The RandomAccessFile is opened using the "rwd" options.

- Locking
The methods update and delete need to lock the record they want to modify.

Lock:
Data class has a Map (implemented as HashMap) of locked contractor
records.
When the lock method is invoked, a check is made to see if the Contractor
already has been locked by another thread (if it can be found in the
locked Map). While it has, the current thread will wait until notified
that it has been unlocked. If not, a Cookie number is generated using the
CookieGenerator class and it is put as the key and the record number is
put as the value in the locking Map. The lock method returns the cookie
number at the end of its operation to the caller.

Process:
The protected processing operation (update, delete) is performed by
providing the cookie returned from the lock method.

Unlock:
The unlock method checks first to see if the locked Map contains the key
given as parameter cookie to the unlock method and the record number as
the corresponding value. If it can then the entry is removed using the
cookie as the key and the notify method is called to notify the change and
call a potential thread waiting to lock the record.

In the lock and unlock methods the locked Map is synchronized on.

- GUI
The main window has a menu-bar with three menu items namely "File",
"Action" and "Help". The File menu consists of the "Exit" item to exit
the application. The Action menu consists of the "Book" item to book a
contractor and a "Show all records" item to refresh the JTable with
a list of all the records. The Help menu has the "Help Contents" item
which is the onlie user guide and a "About" item.
All these items can be accessed using shortcut keys.
As it was demanded in the instructions, JTable was used to present the
data records. For the search functionality, I provided combo-boxes to
select the exact contractor name and/or the exact location from. Two
buttons were provided for searching and booking (unbooking is not allowd
on the UI).

- Exception handling
The exception handling deficiency of the DB API was worked out by having
the DataHelper class wrap any IOExceptions in an unchecked
ContractorException.
All the exceptions are caught at the highest level which is Client GUI and
Server GUI.

- Design patterns used
Interface
Model View Controller (MVC)
Singleton
Factory Method
Business Delegate
Data Access Objects
Decorator



Hope its of some value.

[peter wooster]

Originally posted by A Mesbah:
...
Lock:
Data class has a Map (implemented as HashMap) of locked contractor
records.
When the lock method is invoked, a check is made to see if the Contractor
already has been locked by another thread (if it can be found in the
locked Map). While it has, the current thread will wait until notified
that it has been unlocked. If not, a Cookie number is generated using the
CookieGenerator class and it is put as the key and the record number is
put as the value in the locking Map. The lock method returns the cookie
number at the end of its operation to the caller.
...

Did you really use the cookie as the key to the Map? It should use the record number as the key, since that's what other threads will try to lock.

Did you use notify or notifyAll when unlocking a record? If the wait is on the Map, you need notifyAll, if you wait on a LockedRecord object that contains a queue of waiting threads for that record you could use notify.

Another missing feature in the lock code is preventing sequences such as
client A locks record 1
client B locks record 2
client A tries to lock record 2 but has to wait
client B tries to lock record 1 = deadlock

This can be prevented by restricting clients to a single lock or by requiring that locks be requested by the client in a specified order, such as ascending. There are more complex algorithms that can detect the forth step in the sequence above, but they are probably excessive. To do this requires a queue of waiting clients for each record, if a client wishes to lock an additional record, it must first check that no other client that is waiting on one of its locked records already has the desired record locked.

[James Turner]

Hi Guys,

I have been following this post, it is very interesting. I am working also on the Bodgitt & Scarper assignment. I have it almost ready to submit except for my locking code.

My locking works as it is but it locks the entire database, I would like to change this to allow clients to lock individual records, that way any number of clients can access the database to use different records.

I am not too sure how to do this without writing overly complex code.

The lock method would need to be synchronized, then if the required record is already locked, the thread can be entered into a queue, waiting for the record. When the record is unlocked the threads on the specific queue can be woken with a notify.

Trouble is, if the lock method is synchronized, and the thread is then put on a queue to wait for a record the lock method will remain locked to that thread as the thread never relinqueshes it's lock on the 'lock' method. This causes deadlock.

If I do not have the lock method synchronized then there is no safe way to check if a record is locked or not as things are not being done attonomously.

Has anyone got any ideas on this issue or any ways i could do it.

Thanx for any help.

Much appreciated.

James.

[Daniel Simpson]

Originally posted by A Mesbah:

Lock:
Data class has a Map (implemented as HashMap) of locked contractor
records.
When the lock method is invoked, a check is made to see if the Contractor
already has been locked by another thread (if it can be found in the
locked Map). While it has, the current thread will wait until notified
that it has been unlocked. If not, a Cookie number is generated using the
CookieGenerator class and it is put as the key and the record number is
put as the value in the locking Map. The lock method returns the cookie
number at the end of its operation to the caller.

Originally posted by peter wooster:


Did you really use the cookie as the key to the Map? It should use the record number as the key, since that's what other threads will try to lock.

Did you use notify or notifyAll when unlocking a record? If the wait is on the Map, you need notifyAll, if you wait on a LockedRecord object that contains a queue of waiting threads for that record you could use notify.

Another missing feature in the lock code is preventing sequences such as
client A locks record 1
client B locks record 2
client A tries to lock record 2 but has to wait
client B tries to lock record 1 = deadlock

This can be prevented by restricting clients to a single lock or by requiring that locks be requested by the client in a specified order, such as ascending. There are more complex algorithms that can detect the forth step in the sequence above, but they are probably excessive. To do this requires a queue of waiting clients for each record, if a client wishes to lock an additional record, it must first check that no other client that is waiting on one of its locked records already has the desired record locked.

You brought up a good point, Peter, when you talked about

Another missing feature in the lock code is preventing sequences such as
client A locks record 1
client B locks record 2
client A tries to lock record 2 but has to wait
client B tries to lock record 1 = deadlock

But is that even possible for that to happen when booking a contractor? Then again, I guess one could debate if the network connection is slow, it might take a while to process, but is it possible for a client to try and book two records that fast? I don't think this would be possible in local mode, as locking-booking-unlocking happens in less then a second. Am I wrong in assuming this? Is it necessary to implement the locking above so a client can have only one lock at any time? Or....what if I made my adapter book method synchronized? That would hurt performance as many could be connected but no concurrent bookings or would that be violating or defeating the whole purpose of network mode?
[ January 05, 2005: Message edited by: Daniel Simpson ]

[peter wooster]

Originally posted by Daniel Simpson:


But is that even possible for that to happen when booking a contractor? Then again, I guess one could debate if the network connection is slow, it might take a while to process, but is it possible for a client to try and book two records that fast? I don't think this would be possible in local mode, as locking-booking-unlocking happens in less then a second. Am I wrong in assuming this? Is it necessary to implement the locking above so a client can have only one lock at any time? Or....what if I made my adapter book method synchronized? That would hurt performance as many could be connected but no concurrent bookings or would that be violating or defeating the whole purpose of network mode?

[ January 05, 2005: Message edited by: Daniel Simpson ]

That scenario can't occur on B&S or UyB if all we are discussing is the book method, since the client never locks more than one record. It is possible if someone were running an arbitrary test of your locking code since the interface doesn't preclude locking multiple records. Also it is stated that we should be building a framework for future enhancement, and booking multiple rooms or contractors is common. Also the examiners may run tests that do that to determine if they can cause an deadlock.

At the very simplest, you should throw an exception if the client tries to lock a record when they already have one locked, note that a lot of simple solutions deadlock if you try to lock a record you already have locked. The second choice of insisting on locking in ascending order gives a bit more flexibility, and the optimal choice is to detect deadlock if it occurs and only throw an exception in that case.

[Omar Kalaldeh]

congratulations



and all the best

[Inuka Vincit]

congrads...

sorry to digress... but is the details posted in certmanager...
I checked today I have passed but I cant find the darn details. I know it used to be posted in the certmanager but the new look doesnt seem to have the details...

well anyways... congradulations

[Dieskun Koper]

There seem to be two databases. The detailed score can be found here:
http://www.certmanager.net/sun_assignment/

[Inuka Vincit]

thanks man found it

[A Mesbah]

Originally posted by peter wooster:


Did you really use the cookie as the key to the Map? It should use the record number as the key, since that's what other threads will try to lock.

Did you use notify or notifyAll when unlocking a record? If the wait is on the Map, you need notifyAll, if you wait on a LockedRecord object that contains a queue of waiting threads for that record you could use notify.

lock method(renNo):

 
           ...
           synchronized (LOCKED_RECORDS) {
                while (LOCKED_RECORDS.containsValue(recNoInteger)) {
                    try {
                        LOCKED_RECORDS.wait();
                    } catch (InterruptedException e) {
                        ;
                    }
                }
                cookie = CookieGenerator.getNextCookie();
                LOCKED_RECORDS.put(new Long(cookie), recNoInteger);
            }
          ...

unlock method(recNo, cookie):

 
       ....
       synchronized (LOCKED_RECORDS) {
            if (LOCKED_RECORDS.get(cookieLong).equals(recNoInteger)) {
                LOCKED_RECORDS.remove(cookieLong);
                [B]LOCKED_RECORDS.notify(); [/B]
            } else {
                throw new SecurityException
            }
        }
        ....

I should have used notifyAll instead of notify now that I look at it, you are absolutely right.

Another missing feature in the lock code is preventing sequences such as
client A locks record 1
client B locks record 2
client A tries to lock record 2 but has to wait
client B tries to lock record 1 = deadlock

This can be prevented by restricting clients to a single lock or by requiring that locks be requested by the client in a specified order, such as ascending. There are more complex algorithms that can detect the forth step in the sequence above, but they are probably excessive. To do this requires a queue of waiting clients for each record, if a client wishes to lock an additional record, it must first check that no other client that is waiting on one of its locked records already has the desired record locked.

I defined a contract in the API for deadlock prevention:
"Any potential users of the locking API should invoke lock/process/unlock as a sequence within the context of a single method call which guarantees it will happen within a single thread of execution."

[Daniel Simpson]

Originally posted by peter wooster:
At the very simplest, you should throw an exception if the client tries to lock a record when they already have one locked, note that a lot of simple solutions deadlock if you try to lock a record you already have locked.

I have noticed that my program deadlocks if I try to lock a record that I have already locked. I can't, however, figure out how I can check to see if the client is trying to lock a record that they have already locked. Any ideas?

public synchronized long lock(int recNo) throws RecordNotFoundException {
        try {
            checkRecordValid(recNo);
            
            Long cookie = new Long(recNo);
            
            while (lockCookieHash.containsKey(cookie)) {
                try {
                    wait();          
                } catch (InterruptedException inEx) {
                }
            }
                
            /* Performs record validity again because record 
             * state could have changed while the thread was 
             * waiting for the lock
             */
            checkRecordValid(recNo);
 
            // put lock cookie into hashmap
            lockCookieHash.put(cookie, cookie);
            
            return cookie.longValue();
            
         } catch (RecordNotFoundException rnfEx) {
             throw new RecordNotFoundException(rnfEx.getMessage());
         }
    }     

[peter wooster]

I have noticed that my program deadlocks if I try to lock a record that I have already locked. I can't, however, figure out how I can check to see if the client is trying to lock a record that they have already locked. Any ideas?

You can do this in the session object (probably the instance of DBAccess) or in a lock manager. I do it in the lock manager by keeping a map of sessions and their pending and active locks. That way you can enforce one of the following rules:

simplest - a session can only have a single lock request at any time

medium - lock requests must be done in a specified order, eg. ascending, this assumes that the resources implement Comparable.

hard - lock requests may be done in any order, but a sequence that would block is detected by checking if session that already has the desired resource is waiting for a resource that this session has locked.

[Daniel Simpson]

Originally posted by peter wooster:


You can do this in the session object (probably the instance of DBAccess) or in a lock manager. I do it in the lock manager by keeping a map of sessions and their pending and active locks. That way you can enforce one of the following rules:

simplest - a session can only have a single lock request at any time

medium - lock requests must be done in a specified order, eg. ascending, this assumes that the resources implement Comparable.

hard - lock requests may be done in any order, but a sequence that would block is detected by checking if session that already has the desired resource is waiting for a resource that this session has locked.

Hey Peter, I'm using sockets in my implementation. What about this idea? What if I used a hashmap that used the record number and the thread's name for the key-value? That way, I can check to see if that thread already is locking a record. Such as it would put into the map- key: 27, value: Thread-1. Then I could do manager.containsValue(Thread.currentThread().getName()) (thus preventing 1)locking multiple records and 2)preventing trying to lock the same record multiple times) what do you think?

[ January 10, 2005: Message edited by: Daniel Simpson ]
[ January 10, 2005: Message edited by: Daniel Simpson ]

[John Winstanley]

How would you keep track of the cookie?

[peter wooster]

Originally posted by Daniel Simpson:
Hey Peter, I'm using sockets in my implementation. What about this idea? What if I used a hashmap that used the record number and the thread's name for the key-value? That way, I can check to see if that thread already is locking a record. Such as it would put into the map- key: 27, value: Thread-1. Then I could do manager.containsValue(Thread.currentThread().getName()) (thus preventing 1)locking multiple records and 2)preventing trying to lock the same record multiple times) what do you think?

You can use the thread if you know for a fact that your socket based networking always uses the same thread. With RMI this isn't the case so the thread is not a good way to represent the client. I define a Session interface that is implemented by the Data class. Each client has a seperate Session, the lock manager stores the resource in a Map keyed by session. When waiting, you need to store the thread as well, and if you use cookies that aren't session ids you will need to store those as well.

[Andy Zhu]

back to A Mesbah's original post of his locking codes. There is no code segment about re-checking the record validity after exiting the while loop and before doing other operation on that record. This might be the cause of 44/80 marks on locking. Note, this issue has been well documented by Anton in this forum.

In Daniel's code segment, he has this check both before and after the while loop. So I think this will be fine wrt the aspect of locking, and is not to address the deadlock (that is separate).