URLyBird 1.3.3 - a few questions

[Kuba Kowalewski]

Hi!

I am preparing for the developer's exam and I would like to ask a few questions. Thanks for any help!

Regards,
Kuba

---

Major questions:

1. I assume that the database is tested via the data access class (Data.java) implementing the DBMain interface. This means that testing software is accessing the database via specified interface. Am I right?

2. How about locking the database in the DBMain? I guess that the locking should not be included in the create/read/update/delete methods but only in the lock/unlock/isLocked methods. Can I assume that the testing software always perform:

{
lock (x);
update (x, newData);
unlock (x);
}

The other option is to include locking mechanism inside data access methods (CRUD) but in this case defining lock/unlock methods seems pointless to me. On the other hand it is nowhere said in my instructions file that the record can be deleted or updated only if it is locked.

3. Are the records in the database added/updated (I mean updating as for example changing the location field not changing the owner field) via the client program? My client program specification does not mention this criteria. Is it just functionality that has to be implemented for the server not for the client?

4. Is the rmiregistry run by the examiner?

5. Are these recNo numbers in the DBMain interface just positions of a record in the database (0th, 1st, 2nd, ...)? I find no other explanation.

6. What is the relation between reserved and deleted records? Can I assume that a record is deleted when it's past date? All record in my database file are from the past. Do I delete at all or is it just "empty" functionality for the server that the client should not bother?


Minor questions:

1. Do you find it a good idea to make the Data class (data access class) a singleton?

2. Should I read database structure from the database file of just copy the schema from the documentation? They are the same, of course. Which one do you find more easily understood by junior programmers :-)

3. What is the name of the submission jar file? I guess it can be anything since I can upload only one file.

[Barry Gaunt]

Hello "Kuba"-

Welcome to JavaRanch.

On your way in you may have missed that we have a JavaRanch Naming Policy for displayed (screen) names. Your displayed name must consist of a first name (or an initial), a space, and a family name (in that order) and not be obviously fictitious. Since yours "Kuba", does not conform with it, please take a moment to change it, which you can do right here.

Posters with nonconforming displayed names will be locked out of JavaRanch after a few posts using those names.

Thanks
-Barry

(NR) - search tag

[Anthony Bull]

Hi, I am currently doing the UrlyBird developer project also, and have nearly completed it (user guide, choices, javadoc and unit testing to go). Here's some of my thoughts on your points.

Major Questions:

1. I have assumed the same thing, that automated tests get run against the Data class. From what I've read, these tests are run before your actual programs are even tried out. For this reason, the DBMain interface should be all thats implemented for the public interface of your Data class, nothing more. For your testing you should write similar unit test cases to exercise your Data.java (that are separate from your server code).

2. Only lock the db via the lock/unlock/isLocked methods. Putting locking inside the CRUD methods turns it from being an interface to the database to having business behaviour. The reasoning behind this is that the javadoc for these CRUD methods does not mention locking, and it doesnt allow people using the code to choose the locking strategy/isolation levels to use for a database.

3. No, the client app only allows changing the owner field of a row. Creation and deletion is not required through the client. Creation and Deletion need to be only implemented in the Data class. I assume that the marker will use an automated program to populate the database with data to use when they are testing your client, using the methods in your Data class.

4. For RMIRegistry it is easier to just get your startup code for the server to look for a registry and start one up if none is running. Then check that a URLyBird server is not in the registry, and if so bind your server instance to the registry. This is only a few lines of code to do. It is best to make things as easy as possible for the marker - think of them as a customer using your app - it should be simple to run.

5. Yes. I've used these as identifiers for a room in the database, as they are unique. Its probably up to you whether you use them or not. I assumed that there could be more than one room at the same hotel available through the application, so there was no other way of uniquely identifying a record.

6. Not sure what you mean about reserved records. In my implementation, the only deleted records are ones specifically marked as deleted. All data is in the past in my DB, but they are still valid records in the database. They can still be searched for by my client. They just cannot be booked by the client due to the 48 hour rule. In a full, real-world server implementation there would be a background job that deletes these records as they expire.

Minor Questions:

1. I didn't make it a singleton. I have a business model class (BusinessServiceModel) that implements the business rules of the server and talks to the Data class. The Remote service (RMI) impl has an instance of this BusinessServiceModel to service any calls from clients. As there is only ever one instance of a remote server in the registry, there will only ever be one Data instance and one BusinesServiceModel instance, making it a singleton without needing code. The Data instance is kept for the lifetime of the BusinessServiceModel instance. Also to programatically make it a singleton means having extra public methods that aren't on the DBMain interface which isnt a good idea.

2. This is one I struggled on. I originally got it to read the structure and then use those indexes to find what column was what. This required either (a) my Data class to know about the structure of the database rows or (b) my Data class to have extra methods on it for extracting the schema so my BusinessServiceModel could learn the structure. Both these options seemed wrong, especially (a), as it is a database impl, and should never care about what data it is storing. (b) seemed wrong too, as the interface they had provided in DBMain does not allow for this. So the solution is to document this in my choices file, and to hard code my BusinessServiceModel to know that when it gets a String[] record, that index x gives me column x from a row.

Ok, thats about it. Hope it helped - if you have other ideas let me know, I am always keen to alter my design if it makes it better.

[Kuba Kowalewski]

4. For RMIRegistry it is easier to just get your startup code for the server to look for a registry and start one up if none is running. Then check that a URLyBird server is not in the registry, and if so bind your server instance to the registry. This is only a few lines of code to do.

Very clever! I did not think about this. I guess you mean to use classes from java.rmi.registry?

5. Yes. I've used these as identifiers for a room in the database, as they are unique. Its probably up to you whether you use them or not. I assumed that there could be more than one room at the same hotel available through the application, so there was no other way of uniquely identifying a record.

Since you mentioned automated population of the database I think it does not matter how you recognize your records since create method returns newly created record's. On the other hand identifying records by their position in the data file is the most natural...

Minor Questions:

1. I didn't make it a singleton. I have a business model class (BusinessServiceModel) that implements the business rules of the server and talks to the Data class. The Remote service (RMI) impl has an instance of this BusinessServiceModel to service any calls from clients. As there is only ever one instance of a remote server in the registry, there will only ever be one Data instance and one BusinesServiceModel instance, making it a singleton without needing code. The Data instance is kept for the lifetime of the BusinessServiceModel instance. Also to programatically make it a singleton means having extra public methods that aren't on the DBMain interface which isnt a good idea.

I guess it is not wise to make the singleton since the automated tests propably try to access data by the default no argument constructor. Now I get it.

2. This is one I struggled on. I originally got it to read the structure and then use those indexes to find what column was what. This required either (a) my Data class to know about the structure of the database rows or (b) my Data class to have extra methods on it for extracting the schema so my BusinessServiceModel could learn the structure. Both these options seemed wrong, especially (a), as it is a database impl, and should never care about what data it is storing. (b) seemed wrong too, as the interface they had provided in DBMain does not allow for this. So the solution is to document this in my choices file, and to hard code my BusinessServiceModel to know that when it gets a String[] record, that index x gives me column x from a row.

I think that because of:
"the data must continue to be manipulated for reports using another custom-written application, the new system must reimplement the database code from scratch without altering the data file format" it is highly acceptable to hardcode the database structure. Anyway it is always quicker.

And: "The IT director does not anticipate much reuse of the first Java technology system, but intends to use that system as a learning exercise before going on to a web based system." Which possibly means that if they redesign the database it will happen no sooner than at the point of moving to the web based system.

By the way: do you validate the data file when the server starts?

Thanks a lot for your help!

Best regards,
Kuba

[William Smith II]

I just got another quick question about locking, that is related to this thread.

Is it OK to call the lock(), unlock() in the create() method?

The reason I do this is because when a call to create() is invoked, the business rules won't know what the next recNo is, and probably shouldn't. So what I do is I get the highest recNo and add one (using syncronized code on an Integer), then I call lock(recNo) (synchronized within my lock manager), then I create the record (again syncronized).

Is it OK to do this?

[R van Vliet]

Hi William,

I agree that the business layer shouldnt know (or care) what the next recNo for a created record should be. I'm assuming your issue with this is that in order for the business layer to lock the record it has to know the recNo which obviously is unknown before creating the record.

In my opinion this issue shouldnt be solved by locking inside the create method because it's not a record locking issue but a recNo/database locking issue.

I think your assumption that the next recNo is just the highest recNo plus one is not correct since (at least in my instructions) it's specifically stated you should reuse previously deleted records. Or at least, it's hinted that deleted records should be available for reuse. The only way to find the next recNo is to access the database file and find the first record that either is marked as deleted or append it to the end of the database file if no deleted records are found.

If the above assumption about having to reuse deleted records is correct, then even if you did lock inside the create method, determining the next valid recNo happens outside any sort of record lock, so if two threads do that part at the same time they'll both attempt to create the a record with the same recNo. Obviously this isnt fatal because the first thread will succeed in doing so and the second attempt will throw the DuplicateKeyException.

If you make a seperate synchronizing system for acquiring the next valid recNo you could avoid this issue (in other words, make sure that the method that finds the next valid recNo is synchronized against createRecord so that this method will always block until the createRecord method returns).

I find the above a preferable approach, because in the scenario where two threads try to create a new record in the first approach one will succeed and the other will fail (DuplicateKeyException) whereas in my approach both will succeed. Admittedly the latter approach is more complicated in nature.

All that said it's entirely possible i misinterpreted the instructions or am overcomplicating things

That's my $0.02 though

[William Smith II]

My DBMain says that it can possibly re-use an existing record.

// Creates a new record in the database (possibly reusing a
// deleted entry). Inserts the given data, and returns the record
// number of the new record.
public int create(String [] data) throws DuplicateKeyException;


I'm also not seeing any issue with duplicate key. My specs mention nothing about primary key, so I have gone under the assumption that recNo is the only key.

My create(String[] data) method simply does the following (pseudocode), all within Data:
synchronized(highestRecNo) {
newRecNo = highestRecNo++;
lock(newRecNo);
synchronized(on my file) {
create record using newRecNo /* point A */
}
unlock(newRecNo);
}

I just can't find a way of creating a new record without calling lock()/unlock() within my Data class. My Data class does all file I/O and instantiates a LockManager.

Note: I realize that if thread S creates recNo 38 and thread T creates 39, then when they go to write, S could write in the 39th spot and T could write in the 38th spot. I don't see any big issues here based on the limitations of the DBMain interface and lack of suiteable primary key (which is just recNo)

[R van Vliet]

Hi William,

You're correct it's not a must to reuse records, but I definately consider it a should. Not reusing disk space of deleted records would mean your database would need to grow in file size for every new record, regardless of the number of records marked as deleted. I feel this could lose you valuable points, the comment doesnt specify possible reuse for nothing.

That said, I know recNo is the only key, and if you dont plan to reuse records flagged as deleted simply finding the highest recNo + 1 means you'll never have a recNo collision as you indicated. So assuming you go for that approach :

Perhaps our assignments are slightly different, but my lock method looks like this :

public long lockRecord(long recNo) throws RecordNotFoundException;

Note the RecordNotFoundException. My lock method must fail if it cannot confirm the record actually exists. This check will fail until you write the record to the database, so any attempt to lock on a record you are about to create should fail i think.

And even if it didn't, what kind of problem is your record lock fixing? The worst case example scenario here is thread S trying to create a record and thread T guessing what recNo it'll be and try and update it at the same time. Again updateRecord throws a RecordNotFound exception if it cannot find the record in the file and since you have your file instance synchronized this seems sufficiently thread safe already (either the create finishes completely before the update method can determine record existence, or the update method throws a RecordNotFoundException before the other thread creates a new record).

Perhaps I'm missing something, in which case I'd like to hear about it
[ November 16, 2007: Message edited by: R van Vliet ]

[William Smith II]

I think because recNo is the key, the create() must be atomic.

The situation, while will create correct results, but is not good programming:
Thread S reserves recNo=38, blocks.
Thread T reserves recNo=39
Thread T creates record, what should be 39, in the 38th spot.
Thread S resumes and creates record 38, in the 39th spot.

In my URLyBird assignment, that's OK, but I think it's best to leave create() as an atomic operation. In other words, the create() method should call lock()/unlock(), or there are no guarantees of thread safety.


I have another question, and searching isn't giving me a good answer.

Whom should be invoking the lock() and unlock() calls for read(), delete() and update()? Right now, my assignment works. But I have the locking done in the business layer (thick client).

I'm at the point where I'm considering putting in a Facade for my Data class and placing the locking in there. A DataFacade will pass all functions onto Data plus ensure locking is done correctly. My other option is to keep the locking in the business layer, but I don't like that because a database should not leave locking up the client, I don't think. To me Data is a low-level class of I/O operations.

[R van Vliet]

William,

I dont agree with your first example nor with the fact that the entire create method must be atomic. If you determine in the create method the new recNo is 39, you should write the new record at offset 39 * recordSize, and not at the end of the file. That is indeed broken. Whether or not determining the next recNo and the actual file update is atomic is not really that relevant I think.

[R van Vliet]

I forgot to address your second question

Yes I also use a facade as the public interface to my client with methods such as bookRoom, where bookRoom looks roughly like this (pseudo) :

    
lockRecord(room.getRecNo());
room.setCustomerId(customerId);
updateRecord(room.getValues());
unlockRecord(room.getRecNo());

As you can see I currently use a Room value object here which is only used by the facade class and in the client MVC components.

EDIT : To clarify, there's a step between the lock and the setCustomerId that verifies if the room has been modified between filling the value object (room) and acquiring the lock.
[ November 18, 2007: Message edited by: R van Vliet ]

[Alex Belisle Turcot]

Hi William,

my create is atomic and according to my design it is impossible to lock/unlock deleted records (it returns RecordNotFound).

In other words, only 1 thread can execute the create method and no other thread can lock/unlock the deleted records. The create method can freely alter deleted records or add a new one.

bye,
Alex
[ November 18, 2007: Message edited by: Alex Belisle Turcot ]

[William Smith II]

Hi Alex,

I did not synchronize my entire create() method. I do however have a synchronized block which: determines the next recNo and then calls lock(). That's the end of the synchronized block. So all the threads can create threads in an ordered and friendly way without stepping on each others work.

Another issue I see is that my create() method throws DuplicateKeyException. But that won't be possible since recNo is unique, there's no formal key definition outside recNo.

I think that these issues may be dependant on which assignment that you receive from Sun.