RecordID implementation issue???

[Khaled Mahmoud]

According to the DB interface the client can read a record even if it is booked by another user.
I have the following scenarios

The first case:
Client A locks the record
Client B reads the same record while client A is updating that same record
In this case for example client A migh be updating the first two feilds of the record while client B is reading the second two.We could end up with a situation such as client B reads a record where half of it is valid and the other is not.


The second case:
If the record ID is to be implemented by using the record number (position in data file).The following case could happen

Record Z is located in position 5 and given an id of 5

Client A requested to read record Z and got 5 as its id
Client B books the record to be deleted and deletes the record
Client C adds a new record to the database in the same position as record Z
Client A now wants to update record Z.Client A does not know that record Z has been brutally deleted by Client B and client A will think that the new record is record Z because it is placed in the same position.Client A will think it is updating record Z which was deleted and he didnt know at all about what happend.
I think that the recordID should composed of a mix of the place of the record in the file and its contents.I mean using a hash algorithm that generates a uniqe number from the contents of the record and also combine with that the record number so that we can access the record easily by using the record ID.
What do you think???

[ August 09, 2006: Message edited by: Khaled Mahmoud ]
[ August 10, 2006: Message edited by: Khaled Mahmoud ]

[Daniel Bryant]

Hi Khaled,

Regarding your first problem - this is a big issue and is something that I believe would cause you to automatically fail if you don't guard against this. There are much worse variations of this problem (such as only writing half of a new record) that could leave your data file in an undetermined and corrupted state, which I believe Sun will see as unacceptable. Depending of how you have implemented your file reading a simple synchronization will guard against this happening with only a minor (and very acceptable) performance hit.

Regarding you second problem - that's an interesting solution, but do you think this would be easy to read by a junior programmer? I would recommend simply accepting that "Non-repeatable Reads" (see here for more info on terminology) can occur in your implementation of a database and code accordingly - maybe before updating or deleting a record you should lock the record (preventing any concurrent modifications) and then read the record again and compare it to the original record before it was modified. If they don't match you know you have a non-repeatable read and can inform the user accordingly (i.e. you know that the row has been modified by some else between the initial read and the update, or that the record has been deleted and then reinstantiated with new data)

I hope this helps.

Best wishes,

Daniel
[ August 11, 2006: Message edited by: Daniel Bryant ]

[Khaled Mahmoud]

Hi Daniel

Regarding the "Non-Repeatable Read" I think the application should be able to distinguish between the two cases

1-The record has been modified.And as you said is would a very great idea to tell the user that the record has been modified and perhaps suggest to the user in the GUI part of the application to refresh the current record.

2-When the record has been deleted and another record was inserted in the same place.To consider this case to be the same case as deleted record.

I think it is very important to distinguish between these two because they are logically different.

Another thing i want to take advice for:
At the current stage i am trying to get all the logic for the DBEngine that will access the file.Then start to think about networking issues and then about the G.U.I and do the software design and then coding.What do you think??Actually this is the first time in my life that i work on such a project.What do you think???
[ August 11, 2006: Message edited by: Khaled Mahmoud ]

[Khaled Mahmoud]

For protecting against dirty reads i am thinking of implementing a read manager

A client cannot read a record unit this record is marked in the read manager as free.
When a client books a record and wants to update or delete this record it should mark this record in the data manager as notfree before starting the update process.This way dirty read issue will be solved.
Another thing you said which terrified me so much,which is adding half a record and causing the database file to be corrupted.
For the part of the assignment i have,the mentioned functionality is searching ,booking and updating.
But even if the user is going to add a new record which it could be a future enhancement,i dont imagine how a create record operation would cause the whole datafile to be corrupt by adding only half a record.

[Daniel Bryant]

Hi Khaled,

I think it is very important to distinguish between these two because they are logically different.

That's an interesting point, and something I may think about a little more later on. However, for the moment I am going to treat the two incidents as the same - as an alternative you could have a simple algorithm that determines if the data in more than, say, 4 fields have been changed then assume the record has been deleted and a new one created, but this might not be a good idea. Documenting this could be tricky and ideally you would base such an algorithm on a primary key (which is lacking in the Sun spec )

Another thing i want to take advice for:
At the current stage i am trying to get all the logic for the DBEngine that will access the file.Then start to think about networking issues and then about the G.U.I and do the software design and then coding.What do you think??Actually this is the first time in my life that i work on such a project.What do you think???

Sounds good to me - that's how I approached this (in conjunction with Andrews SCJD for J2SE 5.0 book)

For protecting against dirty reads i am thinking of implementing a read manager

On first read this sounds a bit complicated, and may restrict concurrent access in you application. Imagine you have 30 clients all accessing the server, it could easily get to the point when no records could be read - I think (and there are several posts by Andrew Monkhouse and others which I deduced this information from) that non-repeatable reads are acceptable in an application such as this (and if Sun wants more they should us a RDMS in conjunction with our application )

But even if the user is going to add a new record which it could be a future enhancement,i dont imagine how a create record operation would cause the whole datafile to be corrupt by adding only half a record.

The scenario I was thinking of here, which you will need to guard against, is this:

1. Client A creates a new record (Thread 1 assembles the data and begins to write to the file)
2. Client B also wants to create a new record (Thread 2 assembles the data)
3. Half way through writing the record to the file Thread 1 is moved to the WAITING state (i.e paused) and Thread 2 gains control of the CPU.
4. Thread 2 locates the end of the file (which only has half the record written) and then writes its whole record to the file
5. Thread 1 gains control of the CPU and finishes writing the remaining half of the record

The problem we have now is that the DB file is corrupt and there are at two records that can't be read properly. However, this situation may not occur depending on which method you have used to access the DB file, or the method you have used to calculate where the new record should be written.

I hope this helps.

Daniel
[ August 12, 2006: Message edited by: Daniel Bryant ]

[Khaled Mahmoud]

Hi Daniel

For the read manage i suggest i didnt mean non-repeatable reads.The aim
of the read manager is to guarantee that a client cannot read a record while it is being updated.That's any thread that wants to update a selected record it has to mark that recotd as non-free so that no other records can read the record while it is being updated.Also prevent a given thread from updating a record while it is being read.I think this would completly protect from dirty reads.Taking into consideration that the update processes is much less than read and takes less time as i think.

For the adding process i am also thinking of implenenting a read manager.Any client who wants to add a new record to the file has to ask the record manager for an empty record the read manager allocates space in the file for the that thread an returns the location in the file where to insert that new record.This way the addition problem is guaranteed not to happen.
For the primary key wheather is included in the speicification or not please read my thread at the following link and tell me your opinion:
https://coderanch.com/t/188507/java-developer-SCJD/certification/Primary-key-update-method-strange

[ August 12, 2006: Message edited by: Khaled Mahmoud ]
[ August 12, 2006: Message edited by: Khaled Mahmoud ]

[Daniel Bryant]

Hi Khaled,

Your idea is very interesting, but I still think it may be complicating the program unnesessarily (This is of course purley my opinion, which may not be correct Does anyone else have any comments?). The main concern I have with your design is that I think you may be unnesessarily restricting concurrency in your server.

For example in my application I only allow one thread at a time to physically read or write to the DB file (to avoid corrupting the DB file), but the actual read and update methods can be called by multiple threads and anything that doesn't involve physical reading/writing (such as constructing the stream of bytes for a record or processing a previously read stream) can be done concurrently. This means the application does suffer from a few non-repeatable reads when there is a large number of concurrent users, but clients are still able to read records efficiently without having to wait for other transactions to finish.

For the adding process i am also thinking of implenenting a read manager.Any client who wants to add a new record to the file has to ask the record manager for an empty record the read manager allocates space in the file for the that thread an returns the location in the file where to insert that new record.

I like this idea (as it offers several benefits, but does not effect concurrent access), and I have implemented something similar in my application.

I'll have a look at the other thread now...

Daniel

[Khaled Mahmoud]

Hi Daniel

For example in my application I only allow one thread at a time to physically read or write to the DB file (to avoid corrupting the DB file), but the actual read and update methods can be called by multiple threads and anything that doesn't involve physical reading/writing (such as constructing the stream of bytes for a record or processing a previously read stream) can be done concurrently.

I think is the scenario i have been looking for.Sun wants simplicity and prefers simple solutions over complex ones.This one is very simple,clear and smart.
The primary key issue needs further investigation.
Solving the primary key issue will consequently solve the issue i mentioned in the first thread which is:

Client A requested to read record Z and got 5 as its id
Client B books the record to be deleted and deletes the record
Client C adds a new record to the database in the same position as record Z
Client A now wants to update record Z.Client A does not know that record Z has been brutally deleted by Client B and client A will think that the new record is record Z because it is placed in the same position.Client A will think it is updating record Z which was deleted and he didnt know at all about what happend.

[ August 12, 2006: Message edited by: Khaled Mahmoud ]

[Thirumurugan Mylrajan]

Nice thoughts on the issues.

In my case
1. I did not consider the issue of dirty reads. There is a very small possibility that the data displayed is corrupted. Even in commercial RDBMS dirty reads can be performed( uncommited read in DB2). You can assume the same scenario.

2. To prevent the data being corrupted during update, lock the record and check the data on the record position with the data that is being displayed. If it had been modified, give a message that the data has changed. Otherwise continue with update of the record.

Thanks.

[Khaled Mahmoud]

Hi Daniel

For example in my application I only allow one thread at a time to physically read or write to the DB file (to avoid corrupting the DB file), but the actual read and update methods can be called by multiple threads and anything that doesn't involve physical reading/writing (such as constructing the stream of bytes for a record or processing a previously read stream) can be done concurrently.

I think a have come up with a solution that will enhance concurrency and will protect against dirty reads.This solution is an enhancement on your inventive and creative idea of allowing only one thread at a time to read to or write to the file.
Which is:
The operations occuring on the file can be divided into two main categories:
1-Read operations
2-Update and delete and addition operations.
The idea is allow only one kind of those two categories to access the file at a time.
What i mean is:At any given moment,many threads are reading from the file or many threads are changing the contents of the file (update,add,delete).
With the implementation of a simple lock and addition manager,we can gurantee that no contradiction will happen.This way for example two threads may be updating the database file,and another two threads are adding records to file at the same time.

What do you think??

[Daniel Bryant]

Khaled,

You could do this - and you might want to look at the new JDK 5.0 ReentrantReadWriteLock implementation of the Lock interface - but I still think the solution may be more complex than it needs to be (and you'll also have to be careful with multiple deletes/updates - what if one thread is updating record 1 and another deleting it?)

Best wishes,

Daniel