Almost ready to hand up: Just cleaning up and have a few questions

[martin naughton]

Hey Guys,
Thanks for any help you can give. I promise when i pass or even before i pass i will post up how i did my project. What i put in and what i left out. hopefully it will help some one in the future.

Question 1 about where to put the locking

I have made my data class a singleton. I did not want to expose all the data operations to the client so i made a interface that only implemented update read find from the data class. These are the only operations the the client will see. I am at the moment trying to figure out where i should locking methods from my data class. Should i have them my data class where it does the direct changes to my database file or have them in the interface i made between the client and database. Putting them in the interface makes sense to me but i am thinking about when Sun want to test it. will they have the same setup as Roberto has in this threads class.
https://coderanch.com/t/427863/java-developer-SCJD/certification/Tests-Data-class-locking-mechanism

where he makes the data instance and then calls lock read unlock himself. Like they are not expecting to call read in the data class itself and it handles all the locking and unlocking in that method.

Sorry if it sorry confusing.


Question 2 about the find method on the server.
This is what i was told

// Returns an array of record numbers that match the specified
// criteria. Field n in the database file is described by
// criteria[n]. A null value in criteria[n] matches any field
// value. A non-null value in criteria[n] matches any field
// value that begins with criteria[n]. (For example, "Fred"
// matches "Fred" or "Freddy".)
public int [] find(String [] criteria)
throws RecordNotFoundException;

An example of how my find method works in my Data interface. If i give the criteria Palace and Smallville my find method will return these combinations of results.

Palace Smallville
Castle Smallville
Excelsior Smallville
Palace Whoville
Palace Hobbiton

Is this correct or should it only return exact matches so my find method should return only one result. In my GUI i make sure it is only the exact match that appears. In the GUI only one match will appear.

Also if i give it all null values like this in my test case. It will return no record. Just wanted make that is what they wanted.
@Test(expected = RecordNotFoundException.class)
public void testFindNothing() throws RecordNotFoundException {
DBMain data = Data.getInstance(dbCopyLocation);
data.find(new String[]{null,null,null,null,null,null,null});
}


Question 3 about how locking should work
if i have my three methods delete, update and read that should implement locking before using the methods and then unlocking after successfully using the methods.the locking is only done on the record number. three threads should not be able to one should get the lock and update, after that the next thread should get the lock and delete then the next thread should throw an exception because the record does not exist to read. I understand those perfectly.

My only problem is while these three threads are running then create and find are running free. Nobody is stopping them from doing as many finds or creates. The only way i think of stopping them running free is by using synchronized block in their methods. Since the data interface is a singleton there can only be one create and one find happening at one time. but there problem i see is that one of the read update and delete could be running at the same time as a create or find. I think this might present a problem but have not found what it might be yet. I was thinking that if only one of the 5 operations (create,read,update,find, delete) could run only be running at one point in time then the data class would be thread safe and stop corrupted data.but i am not sure how to bind all 5 methods together. i could use ReentrantLock() so no other thread could access the data class but also i am not sure if i should bind all 5 operation. Will it cause a problem if one of create update and delete are running at the same time as create and find?


Question 4 about reusing deleted records.ANSWERED
I actually just figure this out while i was writing this. I had a problem when i wanted a new record number. i was record.size. to get a new number. but say if the size was 30 and i removed the first 5 records. When i called record.size +1 to get a new record number i would get 26. But that record still exist in record so i would get a duplicate exception. Instead i have now implement this

 record.size + deletedrecords.size+1

. deletedrecords map will contain the 5 deleted records and records will contain 25 records so i would get a new record number of 30.

I also am reusing deleted record numbers and positions. i keep them in deletedrecords. if deletedrecords is empty then create a new record. if it contains some thing then use it. Easy.


thanks to anyone spending the time to help me.


also roberto i put the names and id of the threads everytime i started one. Then in debug mode i could how may of each thread was running. with just the ids of the thread it is hard to see what they are doing. For example i could see loads of threads with the name UpdatingRandomRecordThread. then i knew that is where my deadlock was. Just a suggest. i can post my updated one in your post.
https://coderanch.com/t/427863/java-developer-SCJD/certification/Tests-Data-class-locking-mechanism


thanks
martin

[Roel De Nijs]

Hello Martin,

First a general remark: you should only lock/unlock records when calling delete or update, not read.

1/ your Data class should be thread-safe, so it can be used in a multi-threaded environment (and your database file does not get corrupted). As a 2nd remark: you have to call lock-method before you make a call to update/delete method and finally you have to call unlock-method. An update/delete should only occur by the client that has successfully locked the record. Only 1 client at a time can lock a record, all other clients have to wait until the other client calls unlock, before trying to lock that record again.
Calls to lock/unlock should not be called inside the update/delete methods, but should be called seperately (in your business service).

2/ my find-method returns the same set of records. My GUI only shows exact matches. When I pass all null values to my find-methods, all records will be returned (but that's just my interpretation of the requirements). But returning all records is one of the must requirements if I'm not mistaken, so I'm curious to know how you implemented that one.

3/ because all my methods are synchronized, only 1 method can execute at a time. But of course an update of record 3, followed by a delete of record 5, create of a new record,... should occur without any problem, without corrupting your database file. You could use synchronized blocks, a class from the new concurrency API or simply mark every method as synchronized (the simplest approach possible)

4/ Glad to hear you were able to solve your problem yourself

Kind regards,
Roel

[martin naughton]

Roel De Nijs wrote:
First a general remark: you should only lock/unlock records when calling delete or update, not read.

Thanks i need to make the read method then synchronized then since i only want one read happening at one time.

Roel De Nijs wrote:1/ your Data class should be thread-safe, so it can be used in a multi-threaded environment (and your database file does not get corrupted). As a 2nd remark: you have to call lock-method before you make a call to update/delete method and finally you have to call unlock-method. An update/delete should only occur by the client that has successfully locked the record. Only 1 client at a time can lock a record, all other clients have to wait until the other client calls unlock, before trying to lock that record again.
Calls to lock/unlock should not be called inside the update/delete methods, but should be called seperately (in your business service).

When you say "only 1 client at a time can lock a record" do you mean that even though different threads are trying to get locks on on different records they have to wait until they are allowed a lock

e.g thread 1 gets lock on record 10
thread 2 attempts to get a lock on record 15 but has to wait until thread one give up the lock. Then it can get a lock on record 15.

Correct?


This is my code for my locking. it seems to be working to me when i try Roberto's locking class. It is the same problem i was having a while ago to understand.

An example: thread 1 comes in to lock record 10.He enters fine but no one else can enter until he goes lock.unlock(); line in lockRecord method. He is kind of like no one can use the bathroom until he is finished:) Thread 1 is finished and happen that he is in the locked map.

Thread 2 enters lockRecord fine. He wants the lock on record 10 but since there is a lock on that already in the map he has to wait. since thread 2 is stuck in the method lockRecord no one else can enter it because lock.unlock(); has not been call in lockRecord . Even if they have a different record to lock.

That is the way it looked like it worked to me. When i tried the method with logging some thing else seemed to happen.Is the scenario above or below here correct?

thread 1 comes in to lock record 10.He enters fine but no one else can enter until he goes lock.unlock(); line in lockRecord method. Thread 1 is finished and happen that he is in the locked map.

Thread 2 enters lockRecord fine. He wants the lock on record 10 but since there is a lock on that already in the map he has to wait. He calls lockReleased.await(); when he is stuck and stays there.

Thread 3 enters find even though thread 2 did not call lock.unlock(). it seems the lockReleased.await() call let other thread into the method. He also wants a lock on record 10 but it is in the lock map so he has to wait.


Then with the unlocking.

thread 1 reaches the unlockRecord method. He removes record 10 from the map and calls lockReleased.signal();. thread 2 and 3 and stuck and they hear this signal. There is nothing saying there who ever is waiting longest goes next. It seems that they will both hear the signal and both go for it. Adding to the map and both threads thinking they got the lock.

Is this the correct way i am thinking.

[edit] do not post complete real code snippets

Roel De Nijs wrote:2/ my find-method returns the same set of records. My GUI only shows exact matches. When I pass all null values to my find-methods, all records will be returned (but that's just my interpretation of the requirements). But returning all records is one of the must requirements if I'm not mistaken, so I'm curious to know how you implemented that one.

Actually i had a method that would get all the valid record numbers from the server and then use the numbers to do a read all the records from the database. Since thinking about it i think the find method should have some way of returning all the records. i have now implement the find method to return everything when it is given all nulls. Thanks for clearing that up.

Roel De Nijs wrote:3/ because all my methods are synchronized, only 1 method can execute at a time. But of course an update of record 3, followed by a delete of record 5, create of a new record,... should occur without any problem, without corrupting your database file. You could use synchronized blocks, a class from the new concurrency API or simply mark every method as synchronized (the simplest approach possible)

I made create,read,update and delete with synchronized blocks. i have synchronized them to the randomAccessFile object. That means that only one of the operation can being operating at one time. I did not synchronized because it gets all it information from read. If find got a lock on randomAccessFile object and then did a call to read method. the read method would not start until find gave up the lock on randomAccessFile object. but find would not end until read finished. Would be dead lock there.



Really really really thanks for your time.

[Roel De Nijs]

A scenario like this should be possible:
t1 -> lock(1);
t2 -> lock(2);
t1 -> update(1);
t3 -> lock(1); // t3 has to wait until record 1 is unlocked
t4 -> lock(3);
t4 -> delete(3);
t4 -> unlock(3);
t5 -> lock(1); // t5 has to wait until record 1 is unlocked
t2 -> update(2);
t2 -> unlock(2);
t1 -> unlock(1); // t3 and t5 will "fight" for the lock on record 1

--> only when threads try to lock a record that is already locked, they will have to wait

I made create,read,update and delete with synchronized blocks. i have synchronized them to the randomAccessFile object.

That's certainly a valid approach. It will be more performant than marking each method as synchronized. But the latter approach will be simpler and more easy for a junior programmer to understand

[martin naughton]

thanks again. Nearly finished just little things keep popping up.

Question 1 scenario
2 network GUIs are started in network mode then one changes a booking. It will not appear in the other GUI unless they restart the network GUI and then the entry will show. I know it is not a requirement but is it an implicit requirement that the GUI should be updated if changes are done to the file by another gui?

Question 2 scenario
I seen in the SCJD book that the server will only close down only if all the operations are completed or there is no more operations. Is this implicitly required?


thanks. thought i would had it up today but probably be monday. thanks

[martin naughton]

Also i was going to leave out logging in my application. instead of i have error message popping up if there is problems. this message a user can understand because they are simple. I have documented in my design choices why i left out the logging but their argument might be the junior programmer might find it hard to debug?

Is this right or wrong?

[Roel De Nijs]

1/ That's confusing me. You have to restart an application to see a booked entry What about refreshing the data doing a search? My gui is also not updated when another gui has made a change to the database file (e.g. a booking), but if this gui executes a new search, the booking will be visible

2/ It's not implicitly required to first end all operations before closing down the server, but you have to make sure your server is not shut down in the middle of a write operation (might result in a corrupted database file)

3/ I kept the logging I used during development in my code, but disabled all logging. I justified this decision in my choices.txt: these logging statements can be simply turned on and can be used by debugging the application when a bug is found or a new requirement is implemented

[martin naughton]

Roel De Nijs wrote:1/ That's confusing me. You have to restart an application to see a booked entry What about refreshing the data doing a search? My gui is also not updated when another gui has made a change to the database file (e.g. a booking), but if this gui executes a new search, the booking will be visible

Yes when a user does a search it will come up in the results. just the GUI is not updating every second i mean. So that is ok.

Roel De Nijs wrote:2/ It's not implicitly required to first end all operations before closing down the server, but you have to make sure your server is not shut down in the middle of a write operation (might result in a corrupted database file)

I implemented this very easy. i used the static map i had for the locks and checked if that was empty before shutting down.

Roel De Nijs wrote:3/ I kept the logging I used during development in my code, but disabled all logging. I justified this decision in my choices.txt: these logging statements can be simply turned on and can be used by debugging the application when a bug is found or a new requirement is implemented

i implemented the log now.should of done it from the start instead of doing system.out. thanks

Thanks a million Roel. Probably submit it tomorrow or tuesday. just have tidy up the choices and user guide. also have to fully build everything into one jar and do rmic for the rmi. that should all be doable in one day. thanks

Hopefully nothing else will come up.

[Roel De Nijs]

Hi Martin,

Maybe you can use the ant build file I created (see ScjdFaq) to create all necessary files.

Good luck!
Kind regards,
Roel

[martin naughton]

One more thing came to mind. I am not serializing anything in my implementation but the only warnings i have in my workspace are these. Do i need to make these classes serializable even thought i do intend in using them that way. maybe just use the default value eclipse gives me?

Description Resource Path Location Type
The serializable class ClientDatabaseOperationsImpl does not declare a static final serialVersionUID field of type long ClientDatabaseOperationsImpl.java /DatabaseTest/src/suncertify/client line 20 Java Problem
The serializable class DataBaseConnectionFactoryImpl does not declare a static final serialVersionUID field of type long DataBaseConnectionFactoryImpl.java /DatabaseTest/src/suncertify/remote line 19 Java Problem
The serializable class DuplicateKeyException does not declare a static final serialVersionUID field of type long DuplicateKeyException.java /DatabaseTest/src/suncertify/db line 9 Java Problem
The serializable class JTextFieldLimitDigits does not declare a static final serialVersionUID field of type long JTextFieldLimitDigits.java /DatabaseTest/src/suncertify/common line 13 Java Problem
The serializable class MagicCookieException does not declare a static final serialVersionUID field of type long MagicCookieException.java /DatabaseTest/src/suncertify/db line 10 Java Problem
The serializable class RecordNotFoundException does not declare a static final serialVersionUID field of type long RecordNotFoundException.java /DatabaseTest/src/suncertify/db line 10 Java Problem
The serializable class TableModel does not declare a static final serialVersionUID field of type long TableModel.java /DatabaseTest/src/suncertify/client line 13 Java Problem

I changed you build script a bit to work but now it is working. thanks a million. would you accept any enhancements on the script?

[Roel De Nijs]

I just added a default serialization id (the year of birth )

martin naughton wrote:I changed you build script a bit to work but now it is working. thanks a million. would you accept any enhancements on the script?

You can post your enhancements in the thread of the script, so everything stays grouped together

[martin naughton]

I just found out that i could not submit the assignment until i did the exam. thought it would be the other way around. submit the assignment then do the exam. my mistake. Well it gives me another week to work on this thing Nothing can ever be perfect.

Roel: Did you ever find out the answer to your question here: https://coderanch.com/t/428093/java-developer-SCJD/certification/DBMain-extend-or-change#1899916

I had to use an extra interface because i wanted to close the connection to the database file. I made interface DBOpertations that extends DBMain so it is a DBMain.

public interface DBOpertations extends DBMain

public class Data implements DBMain, DBOpertations {

so now i use DBOpertations as my interface because it has the funtionaility of DB Main and a method to shutdown the database file.

I am sure this is ok.

[Roel De Nijs]

Hi Martin,

I created my own interface, extending the given one, and passed. So I got the answer indirectly

Note: class Data implements DBOperations will do just fine (if DBOperations extends DBMain)

Kind regards,
Roel