• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

if there is 1 RAF instance, then why lock??

 
Jon Park
Greenhorn
Posts: 16
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,

If your design consists of only 1 working RAF instance for the whole program(and all methods using this instance is synchronized), it means only one operation(read,write,etc.) can be done at a time. Then what's the point of having logical locking mechanism(lock(), isLocked(), etc.)? Just for the sake of meeting scjd specification? And is this really meeting the concurrency requirement in the spec, when nothing can be done concurrently?

I personally wanted to have 1 RAF instance per client, but I decided not to, since there are limitations on the number of file pointers that can exist at the same time. A pool of RAF instances can work(and I might implement one), but it seems to be complicating the problem.

Thanks in advance.
 
mohamed sulibi
Ranch Hand
Posts: 169
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi;

there are two level of lock in the SCJD,
1- changing the file pointer.
2- logical recrod lock.

when you synchronized the CRUD method, you solve the first issue.
but for the second issue, but for the second issue there are two location
to lock the record to update it , in the client side or in the server side
(note that type two is what it called read for update, as i think).
if you implements the LRL at server side and don't expose DB interface to
client then you can synchronized the read and update methods. but the
specification is more general and may be it is writen to make the client
make the lock. may be

What you think ?

Regards.
Mohamed darim.
 
Jon Park
Greenhorn
Posts: 16
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, Mohamed,

So, when there is only 1 RAF instance, you still have to synchronize the RAF instance, because of the file pointer issue, as you have mentioned. What I'm saying is that this will also solve the logical locking issue. For instance:

client A and client B calls update at record 1 at the same time.

since client A and B both use the RAF instance(and update method is synchronized), only one of the clients can update the record at a given moment. so there is no need to have logical locking implementation. Everything will work fine without the logical locking mechanism. No?
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Who says there must be only one RAF instance?
 
Alexander Duenisch
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
@John Park

I figured the same as you did. Having a single RandomAccessFile and synchronizing it (and you have to synchronize due to the already mentioned filepointer issue) solves the record locking problem along the way. That's why i decided against a single RAF. Instead, each method that accesses the database creates it's own RAF object and only access to the physical file is synchronized.

What do you think?
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Alexander, your design is the most logical option.

Jon Park, the lock mechanism suggested by Sun in the instructions is about a record lock. Conversely, having a lock over the RAF would be like having a database lock.

It was your conclusion to use only one instance of RAF, but it is not part of the requirements.

Can you image the performance issue it would represent that every client trying to access the database would have to wait until the previous client has finished with it, even when the two clients want to operate over different records?

If you only had a RAF instance, then it would be impossible for two or more threads to read or write into the file without synchronizing access to the RAF. Even if they are working over different records.

Therefore, it is logical to conclude that you will need a different instances of RAF per every transaction that you require your database server to perform. This way, you will not move the file pointer while other thread is operating in a different record.
 
Jon Park
Greenhorn
Posts: 16
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Edwin and Alexander,

I was wondering about having a local RAF instance for each method that accesses the DB, too, but won't it still create a problem when there are too many operations going on? (like, 50 clients calling read() at the same time? I know that some in some systems only 32 file pointers are allowed at a time..)

What happens when you try to create a new RAF instance when there are already too many file pointers in use, thus the system won't allow a new one?

Also, how big of an overhead would be created by creating and closing local RAF instance within the methods?

Thanks for your responses!
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

I was wondering about having a local RAF instance for each method that accesses the DB, too, but won't it still create a problem when there are too many operations going on? (like, 50 clients calling read() at the same time? I know that some in some systems only 32 file pointers are allowed at a time..)

What happens when you try to create a new RAF instance when there are already too many file pointers in use, thus the system won't allow a new one?


I do not know of any specific OS that only allows 32 file pointers. I am testing my application under Windows XP and Ubuntu Linux, none of those have that problem. The JDK documentation does not state there is a limited number of file pointers that can be obtained. I must conclude that, unless you know of any specific OS restriction, it ought to be only limited by the available amount of memory.


Also, how big of an overhead would be created by creating and closing local RAF instance within the methods?


Well, you can test the performance yourself. If you feel it is a major issue (which I don't) then you can open a pool of them and make them available to your clients. Evidently, this might complicate your design, because now, you will need to control this pool of resources, and when they become scarce then the clients performance might be impacted. But you could implement this more or less as a connection pooling works for some databases.

Or at least that's what I think. The truth is that I am not following that design. I am opening multiple instances of RAF and I am using some logging to evaluate performance. Up to now it does not seem an issue, but it is difficult to determine unless I can simulate a few dozens of clients trying to access the system simultaneously (which, frankly, I do not think I am going to test).
[ April 29, 2008: Message edited by: Edwin Dalorzo ]
 
Jon Park
Greenhorn
Posts: 16
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Edwin,
I've implemented a pool of RAF instances, but then maybe I shouldn't be too concerned with the number of working file pointers, as you say. Anyhow, thanks for your comments.
[ April 29, 2008: Message edited by: Jon Park ]
 
Alexander Duenisch
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The question remains, if it is possible to have multiple parallel I/O operations on the same file. Let's assume Thread 1 using RAF A and Thread 2 using RAF B to write to different locations of the same file (we can guarantee, that Thread 1 doesn't overwrite Thread 2's data) at the same time. If this weren't possible, then we would have to lock the whole file anyway. I haven't found a conclusive answer to this issue yet.
[ April 30, 2008: Message edited by: Alexander D�nisch ]
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Alexander , I have the impression that the discussion in this thread, as far as I understand was about something else. But I think I may also understand your concern.

The discussion here was: are multiple instances of RAF needed?

If you use multiple instances of RAF, this will guarantee that if two threads are working on different records simultaneously, one thread will not move the record pointer of the other. To avoid this problem, threads cannot share the same RAF.

As mentioned in the discussion above, another viable design could be to have a pool or RAF instances and shared them between the threads as they become available. That if the developer considers that the instantiation of the RAF might be a performance problem.

Another option, also discussed in here is the possibility of synchronizing the access over the same RAF instance, which would be inappropriate due to performance problems, since this would work as database lock.

Nevertheless, Alexander , you seem to be talking about this other thing:

In the instructions file provided by Sun they require a record lock, but it is only required for update operations (update, delete, and create). But not for read operations.

Therefore, a thread could be reading a record while, simultaneously, other thread is updating it. This could imply a dirty read.

This problem is not easy to address, and I think this is the problem you refer to in your post. Am I correct?

Personally, I did not solve this case. So far I consider it a corner case in my design, and perhaps, due to the limitations implied by the requirements provided by Sun I will simply not address it at all in my application. I will document it, though, in the choices.txt file.

I cannot assure, nonetheless, that this is the best approach. Perhaps other can offer more insight over this issue and how they handled it.
[ April 30, 2008: Message edited by: Edwin Dalorzo ]
 
Alexander Duenisch
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No, what i meant is, if it is technically possible to have multiple parallel write operations on the same file. Or will this lead to IOExceptions.
I simply do not know how a RandomAccessFile (or many RandomAccessFiles in this case) behaves, if used concurrently. To be on the safe side, i have locked up everything, because as you have said yourself, a thread reading a record while simultaneously another thread is updating it would be a dirty read. I fully agree to that. I have done it like the following:



This certainly looks a bit awkward, but unless i'm missing something it is necessary to have the Lock. What do you think?
[ May 01, 2008: Message edited by: Alexander D�nisch ]
 
Frederic Thierry
Greenhorn
Posts: 22
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It is possible to have a single static RAF instance. And a lot of people have been iplementing either a singleton Data class or underlying singleton subclasses (LockManager and FileAccess classes). Now if the data class or FileAccess are singletons then you will have a single RAF object whether it be static or not.
 
Alexander Duenisch
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Now if the data class or FileAccess are singletons then you will have a single RAF object whether it be static or not.


Sure, but if this singleton instance is accessed by multiple threads concurrently and the RAF instances are created in the singleton's methods (like in the example above), then you will end up with multiple RAF instances. Just think about it for a minute.
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Alexander, I do not understand your implementation. It seems that you are using both a read lock and a write lock. But you do not combine them.

Therefore, with this implementation, how will you prevent that while one thread is writing a record other thread is simultaneously reading.

With this design, you are simply preventing that two threads read a record simultaneously, which is unnecessary.

Frederick. According to the design you are suggesting. That is a single RAF instance. If two threads are reading different records. How do you prevent that that second thread moves the record pointer of the first thread?

I can only think of syncrhonization over the RAF instance. This would a database lock.

If you have to make sure that only one thread is using your instances of RAF, then you need not a record lock. Do you?

I am just trying to understand your post. How exactly do you control this issue in your suggested design?
[ May 02, 2008: Message edited by: Edwin Dalorzo ]
 
Alexander Duenisch
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

With this design, you are simply preventing that two threads read a record simultaneously, which is unnecessary.


No, with this implementation i'm preventing any other threads from reading or writing while one thread holds the write lock. An arbitrary number of threads can read at the same time. Unlike the write lock, the read lock is not exclusive. For further reading see ReadWriteLock (Java Platform SE 6)
 
Frederic Thierry
Greenhorn
Posts: 22
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Simple Edwin I first off populate a Map with the Records in my constructor. Hence I use a simple readLock around this Map when client try to read the records. Now when someone is updating or doing any sort of writing to the file I have decided to use a writelock. This is to stop the possibility of dirty reads. Plus both find and read methods allow for a quick read of my Map cache. I prefer my users to see the right information and loose some performance (bare in mind that my reading performance is much quicker than reading directly from the file).

You seem to want to allow a user to read record 5 whilst another is updating 6. That is great idea but then how do you stop a user from reading record 6 if this record is also being updated?
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hmm! Interesting! Now I see what you are doing. Still I have some questions.

Alexander, the way this lock approach works is not at record level, though.

This means that if 5 clients asked the read record for records 1,2,3,4 and 5. And a 6th client asked the write record for record 6 at that time, then this 6th client willing to write his record will have to wait until the other 5 read operations have finished. Right?

Also, if three clients want to write different records. Let's say 8,9,10. The third client will have to wait until the other two have finished to get the write lock.

I hope I have not misunderstood you. At any rate, I have the impression that, unless you implement this at record level, this could be a performance issue. What do you think, Alexander?

Frederic, so you upload all the records from the file to the memory? What if the file grows and grows in records? If the database file grows bigger it could be problem. Do you not think so?

[ May 02, 2008: Message edited by: Edwin Dalorzo ]
[ May 02, 2008: Message edited by: Edwin Dalorzo ]
 
Frederic Thierry
Greenhorn
Posts: 22
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok at the moment the file contains 27 contractors. Ok let's say you have a 1,000 contractors present in the file (that is a large number), well from monkhouse's book 1000 records uses about 10MB of RAM. Hence as he points out this should not be a problem on most computers since they have at least 128MB of RAM. I also feel that using this Map then makes my code much neater. Hence, yes as I had stated earlier once a write operation is taking place all other threads will be in a blocked state until the lock is released. The main advantage with the readLock is that if no write operation is in place then all threads are allowed to read and find records.

I am actually at this very moment wondering whether there is a way I could make the "isLocked" method more useful in my code. Maybe the way to go about this would be to call up the isLocked(recNo) method prior reading a record. Any of you guys have any idea how this method could be of use.
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Fredirick:

I think it is a very smart idea.

The instructions provided by Sun does not contain a good deal of information regarding on how the file is populated or how big it can become.

According to your calculations the file might need to have around 12.800 records to occupy the whole 128 MB. At least in the URLyBird database that sounds quite unlikely.


One more question based on some assumption of mine:

In the URLyBird assignment. I was assuming that the file could be populated every now and then with the records of available hotels. This implies that the file may contain old records, of hotels booked days before, or even hotels that were never booked in the past. All these records would be of hotels that now do not comply with the 48-hours rule.

In you cache design, do you filter out all those historical records, which now should not be visible in the GUI, or do you still keep them in the cache? Because I think that if you filter those out of the cache, then it would occupy even less memory, optimizing the resources consumed by the application. Although they could be little. It is also a risk to keep them, because, even the smallest of the systems, with time, will get to have thousands of records there and the cache would be bigger and bigger day by day.
[ May 02, 2008: Message edited by: Edwin Dalorzo ]
 
mohamed sulibi
Ranch Hand
Posts: 169
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi;


Hi, Mohamed,

So, when there is only 1 RAF instance, you still have to synchronize the RAF instance, because of the file pointer issue, as you have mentioned. What I'm saying is that this will also solve the logical locking issue. For instance:

client A and client B calls update at record 1 at the same time.

since client A and B both use the RAF instance(and update method is synchronized), only one of the clients can update the record at a given moment. so there is no need to have logical locking implementation. Everything will work fine without the logical locking mechanism. No?


so after client A finished from the update method the client B come and
override Client A modification !, please check your update method implementation because i think you have the same problem i made when i starting the project,

the update method is just take the record and override the already found
record just, the update not check if the record is booked or not. it is
the responsiblity of the client to first ready to check if the record booked
or not then to update the record with new owner value.

what you think ?
Regards.
Mohamed darim.
 
Alexander Duenisch
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
@Edwin


Alexander, the way this lock approach works is not at record level, though.

Correct, this approach works at file level - the whole file is locked, when the write lock is engaged (at least, if all access to the file goes through the class that encapsulates the lock).

This means that if 5 clients asked the read record for records 1,2,3,4 and 5. And a 6th client asked the write record for record 6 at that time, then this 6th client willing to write his record will have to wait until the other 5 read operations have finished. Right?

Right.

Also, if three clients want to write different records. Let's say 8,9,10. The third client will have to wait until the other two have finished to get the write lock.

Also correct.

I hope I have not misunderstood you. At any rate, I have the impression that, unless you implement this at record level, this could be a performance issue. What do you think, Alexander?

Maybe so. That's what my initial question was all about: "Is it technically possible to have multiple threads write to different locations of the same file synchronously (meaning: there is no risk of something being overwritten) without getting Errors/Exceptions?". Because if this weren't possible, then there would be no way around locking the entire file. This issue has something to do with file system technology (reading/writing files) as well as the internal implementation of RandomAccessFile (which i do not know), not with Java programming in general. I still do not have a conclusive answer to that.

Aside from that, i'm not so sure that there actually is a considerable performance loss in the implementation above (using the ReadWriteLock).
Just consider the following:

In a single CPU multithreaded environment, threads are given timeslices by the thread scheduler of the OS. A thread is running, when it has been chosen by the scheduler and it goes into paused state, when the timeslice has elapsed. So there actually is no real synchronism here. The job to be done by the threads is just being chopped up in tiny little pieces and all the threads now take turns to handle those pieces. With locking you just make sure the task (i.e. the code to execute) isn't split up, but is run en bloc (you make the code in a locked section atomic).

One example to illustrate this. Let's assume i have two tasks to accomplish:
1. Cook a meal
2. Mow the lawn
With regard to our threading issue, there are two ways of handling this:
First, i could cook the meal from start to end and do the mowing only after the cooking is finished. Second, i could shuttle back and forth between the kitchen and the backyard and do a bit of mowing or a bit of cooking each time. The overall time it takes me to complete both tasks should be the same, regardless of which one of the two alternatives i choose (let's assume i don't loose time on the way).

In addition, a thread that is waiting on a lock doesn't consume any timeslices (or CPU cycles as this is called by the assignment) but gives them away to the running thread.

If i put all this together, then i come to the conclusion, that the total performance loss is not so great at all. The only overhead involved is for acquiring/releasing the lock.

Or am i completely mistaken?
[ May 02, 2008: Message edited by: Alexander D�nisch ]
 
Frederic Thierry
Greenhorn
Posts: 22
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Edwin, you are totally right about filtering. I am doing the B&S assignment hence the 48 hours does not apply to my design. But if one decides to delete one of the records then in my code logic I remove this record out of my map.

Alexander, you are totally right if we are running the code on a single processor. It will take about the same amount of time (we have to take into account the overhead for threading but its really small). The scheduler does indeed provide time slices for each thread (kind of some context switching between threads).

On the other hand if a multi processor machine was being used, then each piece of code could be running on a seperate processor hence that could improve the performance. But that I believe is out of scope for the project. I think it is best to write the code for a single processor machine.

Basically, I have also tried to come up with a solution where reading and writing takes place on the record level but has Alexander stated this seems to be currently not possible. If it was, I think someone might have already found such a solution on the forum. Plus I think it would make the code extremely complex. The only thing I am still scratching my head about is the "isLocked" method. I am wondering if there could be a way I could use it to improve my concurrency with the CRUD methods.

By the way Mohamed, I am doing the B&S which also requires us to book a contractor. Now we do need to use the update method for booking so we can write to the "owner" field (might be a different field for Urly assignment). But when it comes to the Actual booking logic (i.e. Contractor or Room already booked so cannot be booked again) I believe this should be implemented on the client side. I think it is best to keep the "Data" class, or its subsystems if you are using a Facade pattern, as generic as possible. Hence my methods just fulfill the requirements described in the given interface and not more. I will build a thick client that will then provide the bussiness logic.
[ May 03, 2008: Message edited by: Frederic Thierry ]
 
Alexander Duenisch
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Basically, I have also tried to come up with a solution where reading and writing takes place on the record level but has Alexander stated this seems to be currently not possible.

I didn't state this was not possible. I just said i didn't have a clue and i would really appreciate, if someone could enlighten me.
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Alexander

One more thing. If your locks work at the database level, which implies that no thread is writing while you read, and no thread is reading while you write, then, how will you satisfy the Sun requirements about implementing a record level lock?

I mean, you could still implement it as they require, but if you already guarantee a database level lock, then the record level lock is completely useless. It would be just a waste of time. Do you not think this may affect your score?
[ May 03, 2008: Message edited by: Edwin Dalorzo ]
 
Frederic Thierry
Greenhorn
Posts: 22
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well I will let my reading side of things be concurrent and I will make sure that the writing is threadsafe (Hence readers will be able to read records in parallel on a multiple processor machine unless a writin process is taking place.). I rather prefer my users to see the correct data and wait a tiny bit longer.
 
Frederic Thierry
Greenhorn
Posts: 22
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So Edwin what you are trying to say is that we should not synchronize the CRUD methods. We should use the lock mechanism for both update and delete (create is not really an issue). That is a good idea but we would then end up with dirty reads.
 
Alexander Duenisch
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
@Edwin

I mean, you could still implement it as they require, but if you already guarantee a database level lock, then the record level lock is completely useless. It would be just a waste of time. Do you not think this may affect your score?

Writing code that doesn't handle IO in a safe and correct manner might affect my score just as well. Seems like i'm caught between the devil and the deep blue sea :-(


@Frederic

We should use the lock mechanism for both update and delete (create is not really an issue). That is a good idea but we would then end up with dirty reads.

In your READ method, you could check whether the record is locked and if so, throw an exception to prevent dirty reads.
[ May 03, 2008: Message edited by: Alexander D�nisch ]
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
@Frederick and @Alexander

Your approaches are very interesting, and I am just challenging your designs with questions because I thought a free pair of eyes might be useful to you guys.

I think I am ready to draw a conclusion. This is just my personal opinion, and as I have said before, since I have not yet passed the evaluation, I cannot conclude that my approach is the best.

@Alexander

Personally, I think your design of using a database lock may have two weak points that you should really explain in the choices.txt file:

1) It makes completely useless the record lock design suggested by Sun in the instructions file.

2) It may have an impact on performance. You agreed that if a client wants to write a record and there are other 5 clients before it reading the file, then it will have to wait till all other clients have finished. Although it may require a really small amount of time, I just simply think the approach is risky.

@Frederick
Your design, as far as I understand, it does not use a database lock, but a record lock, using a ReadWriteLock per every record. This is a much better idea.

I think the use of a cache is a neat idea, although it probably makes the design a bit more complicated. You now need to add and remove things from the cache, synchronize access to it, and then at the end update the underlaying database. Certainly, with this design you can prevent dirty reads. In conclusion, I think you idea is very smart and nifty.

Fortunately, Sun, in the instructions file says one may conclude this is the only server running and making updates to the file. Since they said they are replacing a legacy application, if such legacy application were still running the cache design would have not worked here.

In my case, I do not prevent dirty reads. But if I client got a dirty record, when trying to update, the server rejects the record and informs the client that its record has been updated by other client. Therefore, my approach is not about preventing the dirty reads. I let them happen and then I throw an exception if so.

This, of course, it's a simple approach and I cannot say it is better than others. Once I have submitted my application I will let you know how it turns out.

Good luck to you guys with your designs as well.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic