• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

locking and others

 
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello

I face some questions while writing the choice.txt.

Regarding locking, I have one RAF file(data server bound to RMI once). All crude operations (readRecord, update etc) are done in synchronized context since file pointer is being shared.

I am locking a record logically before update ONLY because of instuctions in specification. Technically there would be no deadlock without logical record locking since the read/write are done atomically.

Now I am not locking a record logically before reading because I interprete it as optional from specification.

Concern is, logical lock or not, at most one thread is going to read or write at any instance !! Is there a way to escape that ??

Do I show save complexity or show wise decision by not locking records before reading ??

I felt, if I do do not lock, the blocking threads would consume recource. Whereas by followin the locking scheme in writing, they release and wait.
 
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Generally, locking applys to writing not reading.
I think it is a requirement, not optional, to lock a record before write it.
Otherwise it would not be thread-safe. How do you implement your locking?
Not sure what do you mean by 'atomic' IO. Simply putting the IO access code may not guarantee the IO operation is thread-safe. That is why lock()/unlock() are required. Right?
 
Ranch Hand
Posts: 1847
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
no, locking and threading have little to do with one another (though locking does help keeping the threading code performant).

You can have locking and still not be threadsafe, and you could provide threadsafe code without locking.

The lock is to prevent one client to attempt a modification to a record while another client is doing so.
That's a far more coarse grained operation than a lock which prevents multiple threads from gaining access to the same program resources.
The logical lock you program on a record can hold against a broad range of resources simultaneously, resources that are completely unrelated to the work that goes on by synchronising on something.
You could for example write code to enable simultaneous reading and writing to the database using a record level lock to prevent simultaneous updating.
You could never do that by synchronising database access on the file, as that would prevent concurrent access to the entire file.
 
Louis Logan
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Jeroen T Wenting:
no, locking and threading have little to do with one another (though locking does help keeping the threading code performant).

You can have locking and still not be threadsafe, and you could provide threadsafe code without locking.

The lock is to prevent one client to attempt a modification to a record while another client is doing so.
That's a far more coarse grained operation than a lock which prevents multiple threads from gaining access to the same program resources.
The logical lock you program on a record can hold against a broad range of resources simultaneously, resources that are completely unrelated to the work that goes on by synchronising on something.
You could for example write code to enable simultaneous reading and writing to the database using a record level lock to prevent simultaneous updating.
You could never do that by synchronising database access on the file, as that would prevent concurrent access to the entire file.



If I synchronize on a separate object (say a hashmap), I can implement concurrent read/write to database.

I agree about the coarse and fine grain stuff with record locking. I can implement concurrent access and prevent dirty write if
I simply do the following

synchronized(some static unrelated hashmap) {
write on the database;
// or read from database
}

Well I guess the above still allows concurrent read/write.


Next..
With record locking can I do without performing the read or write operation
outside synchronized context??


If not, then without wait() implemented, all reading threads would compete and eat resource... so what do I gain by NOT implementing record lock for reading ?
[ December 18, 2006: Message edited by: Louis Logan ]
 
Louis Logan
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Joe Zhou:
Simply putting the IO access code may not guarantee the IO operation is thread-safe. That is why lock()/unlock() are required. Right?



Are lock/unlock methods tpresent to to guarantee atomic IO operation??

I thought they are there to implement logical record locking ONLY.

I can carry out atomic IO put putting code inside synchronized context.
 
Joe Zhou
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
There is a good link to find out what thread-safe means to a program(thread-safe). I think that is not the point. The point is how to implement the lock()/unlock() and how to use them in the IO access for update/delete/add operations.

A synchronized context is 'atomic'(or thread-safe, or any other terminalogy) only if it runs on the same object(refer tot he Sun Java site ofr the behaviors or a synchronized context).

Think that if 'synchronize' can solve all the problems, there is no need to implement the lock()/unlock(). It is not a sense of logic locking. It is something a must to make your code thread-safe.

Hope I am correct.
 
Louis Logan
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Joe Zhou:
Think that if 'synchronize' can solve all the problems, there is no need to implement the lock()/unlock(). It is not a sense of logic locking. It is something a must to make your code thread-safe.

Hope I am correct.



Joe

We are going further away from my question...
I do not see it as above. lock()/unlock() methods in this assignment are not indispensable to make data access thread safe.
But this design may be used to implement thread safe ,concurrent access to database even if you want to break your data access in multiple units. I.e.
Lock() => do a part of data access => say you are pre-empted here ..=> do second part .. => ... => unlock()

Of cource the 'part of data access' stuff had to be done under synchronized context.

Thats the way I see it. Correct me if I am wrong.





My question was,
Can the data access work done outside a synchronized context ever ?
Otherwise my design does not allow more than one thread working on data file at any instant !!
 
Jeroen T Wenting
Ranch Hand
Posts: 1847
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I don't think you need to worry about more than one thread being able to access the file at once.
Just make sure that no synchronised block spans more than a single record read/write operation, which is the smallest unit of work you should ensure is atomic.
So don't synch the entire method when you're reading a set of records, but synch only inside the read loop inside that method.
 
Joe Zhou
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I uploaded my assignment but not received any mark yet. So I am not sure my understanding is 100% correct. Here is way I implemented:

lock():

initialize a lockObj
synchronize(lockObj){
if record is locked{
lockObj.wait();//this blocks current thread
}
}
unlock():
get the lockObj;
synchronize(lockObj){
lockObj.notify();//this will unblock the thread
}

To thread-safe your update():
lock(recNo);
update(recNo);//no need synchronize this block.
unlock(recNo);

It is what I learned for my assignment. It may or may not be correct.
For the reference only. There are many good posts on this forum. You may search them by key work 'synchronize'

The link for the synchronize from Sun Java is at this
 
Louis Logan
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Joe Zhou:
To thread-safe your update():
lock(recNo);
update(recNo);//no need synchronize this block.
unlock(recNo);



Your file pointer would go crazy if the low level write operation in update(recNo) is not in synchronized context
 
Joe Zhou
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
My testing shows OK. You may synchronize the context in your update(). It won't hurt. Good luck.
 
Ranch Hand
Posts: 89
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

My testing shows OK. You may synchronize the context in your update(). It won't hurt. Good luck.



Did you try multi threads updating different records? The threads won't be blocked and will try to update different records at the same time.
 
Joe Zhou
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Mike,

I think you were asking me.

Sure, I tested a lot useing multiple thread and multiple client. All were OK.
If your thread does not block, then you may insert logs to see if the thread holds the lock. If your thread holds the lock then, for sure, lock.wait() will block till it is notified. This is the most interesting part of the assignment.

Joe
 
Joe Zhou
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I forgot to menting that threads updating different records without blocking are fine. A thread should block only if the record it wants to update is locked. I think this is what the assignment asks for.
 
Louis Logan
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Joe Zhou:
for sure, lock.wait() will block till it is notified. This is the most interesting part of the assignment.



Do you expect the crude write method to be atomic ???

What if
Thread 1 writes partially to rec#1
Thread 2 writes partially to rec#2
..
Thread1 comes back to write the rest. Where would be your file pointer ???

try your code with 10 threads and 1 sec sleep. you would notice something ..

[ December 20, 2006: Message edited by: Louis Logan ]
[ December 20, 2006: Message edited by: Louis Logan ]
 
Ranch Hand
Posts: 443
3
Eclipse IDE C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
ok, I'll bite, i did notice this issue in the code but assumed it was just a scetch outline of code rather than an implemenatation and this is a learning exercise so ...

If your thread holds the lock then, for sure, lock.wait() will block till it is notified. This is the most interesting part of the assignment.



Strictly speaking the will should be a can ...

Apologies to those already aware, but due to the issue of 'spontaneous thread wake up' the general rule of thumb is never call a wait outside of a loop. The issue is the wait could just return without a notify, the notify code should set a flag and the wait have a loop round it to test the flag (Available in most good Java books ;-) )

I was reading some of the Java bug reports where some one posted this as a Java bug , but its always been this way and apparently in later JVM's it might happen a bit more due to internal changes/implementations, so I'ed say this is wrong.

Java 1.5 docs ..

As in the one argument version, interrupts and spurious wakeups are possible, and this method should always be used in a loop:



Java JSL ref

<blockquote>quote:
"An internal action by the implementation. Implementations are permitted, although not encouraged, to perform "spurious wake-ups" -- to remove threads from wait sets and thus enable resumption without explicit instructions to do so. Notice that this provision necessitates the Java coding practice of using wait only within loops that terminate only when some logical condition that the thread is waiting for holds."

The JLS is available here:
<a href="http://java.sun.com/docs/books/jls/index.html<hr rel="nofollow"></blockquote>" target="_blank">http://java.sun.com/docs/books/jls/index.html
 
Joe Zhou
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator


What if
Thread 1 writes partially to rec#1
Thread 2 writes partially to rec#2
..
Thread1 comes back to write the rest. Where would be your file pointer ???



Not so sure about this scenario. I think each thread should have it's own file pointer.



As in the one argument version, interrupts and spurious wakeups are possible, and this method should always be used in a loop:



Good point. If waiting is interrupted, it should be a failure and let it be handled by the exception handler. Make sense??
 
Chris Hurst
Ranch Hand
Posts: 443
3
Eclipse IDE C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Not sure if we are talking at cross purposes ...

Re with referece to spurious thread wake up quote :

Good point. If waiting is interrupted, it should be a failure and let it be handled by the exception handler. Make sense??



Just to be exact spurious thread wake up is not an exception that would be a different issue, the thread will just wake up as if it had the lock but doesn't, hence you need a boolean or whatever.
 
Louis Logan
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thread has its working variable that is written back to main memory when it is swapped out (either release a lock or rescheduled by thread scheduler).

So one thread would retain the modified file pointer when it leaves 'run' state.

Its no exception (as mentioned by Chris) to wake up without a notify.
 
Joe Zhou
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am wondering you two are fresh graduate. I see you are using so much theories from school. I feel sad that I have left from school for so long.
What I am doing is coding the project as the specification and testing the code according to the project requirements. I would never finish my assignment if I dig into the theories so deep. Hope this is not too superising.
 
Jeroen T Wenting
Ranch Hand
Posts: 1847
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Not so sure about this scenario. I think each thread should have it's own file pointer.



The file pointer is the position in the file where you're currently reading or writing.
you can't read from/write to a file at more than one position at a time...
 
Louis Logan
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Joe Zhou:
I am wondering you two are fresh graduate. I see you are using so much theories from school. I feel sad that I have left from school for so long.
What I am doing is coding the project as the specification and testing the code according to the project requirements. I would never finish my assignment if I dig into the theories so deep. Hope this is not too superising.



Did you complete school or a drop out?
 
Joe Zhou
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sorry guys this is too far away from the original topic about locking.


The file pointer is the position in the file where you're currently reading or writing.
you can't read from/write to a file at more than one position at a time...



That is right. Can different threads have different file pointers? I think the answer is YES. Back to the locking above, if different threads work on different records and each has it's own file pointer, then there is nothing to worry about since they are writing data to different track/cylinder/segment. Remember that JVM will take care all the stuff about writing your data to the disc(CS303/CS304 "File Structure" right?).

If threads are working on the same record, then there is a risk that the record is over written by the threads. That is why the lock/unlock is required. When a record is locked, there will be only one thread writing data to that record. So this is the reason for the locking(not for the logic). Locking is not to make sure the file pointer working properly. Again file pointer is controlled by the JVM and your operating system, Windows XXX probably. By the other word, how your data/record is written to disc is not controlled by your code. It is by JVM. You don't worry about it.

Now, you see I did complete my school... not dropped out right;-)
 
Mike Ngo
Ranch Hand
Posts: 89
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

That is right. Can different threads have different file pointers? I think the answer is YES. Back to the locking above, if different threads work on different records and each has it's own file pointer, then there is nothing to worry about since they are writing data to different track/cylinder/segment. Remember that JVM will take care all the stuff about writing your data to the disc(CS303/CS304 "File Structure" right?).

If threads are working on the same record, then there is a risk that the record is over written by the threads. That is why the lock/unlock is required. When a record is locked, there will be only one thread writing data to that record. So this is the reason for the locking(not for the logic). Locking is not to make sure the file pointer working properly. Again file pointer is controlled by the JVM and your operating system, Windows XXX probably. By the other word, how your data/record is written to disc is not controlled by your code. It is by JVM. You don't worry about it.

Now, you see I did complete my school... not dropped out right;-)



Sounds like you rely on the operating system to handle file locking, i.e. each thread open the file, update one record, then close the file?
 
Jeroen T Wenting
Ranch Hand
Posts: 1847
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
that would be the only way he could do it, which would yield terrible performance.
 
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
i am just think whether we can use a FileChannel lock mechanism to handle the update and delete of the record.

Something lidat
RandomAccessFile raf = new RandomAccessFile ( "junk.dat", "rw" ) ;
FileChannel channel = raf.getChannel ( ) ;
FileLock lock = channel.lock ( ) ;
try {
System.out.println ( "Got lock!!!" ) ;
System.out.println ( "Press ENTER to continue" ) ;
System.in.read ( new byte [ 10 ] ) ;
} finally {
lock.release ( ) ;
}
}
 
Chris Hurst
Ranch Hand
Posts: 443
3
Eclipse IDE C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi, I think FileLock is nio ??? does anyone have any additional info on if we can use nio/concurrent ... i.e. my book says ...

In the past Sun has displayed information on its web site indicating that use of the NIO packages was not allowed in the SCJD assignment.



.. but then goes on to suggest its changed and that if not explicitly banned in the assignment we can.

Any high scorers out there want to comment on if they used the new(5) nio and concurrent packages ??? I think that might be a big help to the rest of us or in fact if you dodn't make use of these packages and got high scores.
 
Chris Hurst
Ranch Hand
Posts: 443
3
Eclipse IDE C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
eek .. quoted without reference ...

'SCJD Exam with J2SE 5'
 
henry ang
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
According to Sun, we are fine as long as we uses packages from Sun but not external packages.
 
Louis Logan
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi

Shall I fail if I use jdk1.4 (not 1.5) and just test it out with 1.5 ?
[ December 22, 2006: Message edited by: Louis Logan ]
 
Greenhorn
Posts: 17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi all,
I read this topic but it's still not clear to me how to use lock/unlock.

My specification (S&P) says:
"You may assume that at any moment, at most one program is accessing the database file; therefore your locking system only needs to be concerned with multiple concurrent clients of your server"

And the DB interface says (for lock method):
"Locks a record so that it can only be updated or deleted by this client.
If the specified record is already locked, the current thread gives up
the CPU and consumes no CPU cycles until the record is unlocked."


Reading the posts on this topic make me think that concurrent operations on different records are allowed (according to the DBinterface), but in the specification is clearly said that no more that one program can access the database file.

How can these two specifications work together?


Thanks

Marco
 
Ranch Hand
Posts: 7729
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Marco Santinon:

How can these two specifications work together?


Marco



Think about one program with many threads of execution.
 
Jeroen T Wenting
Ranch Hand
Posts: 1847
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
More than one client can access the database server, which is the only application accessing the database file.

And a single client could try to work with several records at a time (though this is not a requirement, you could code for it).
 
Marco Santinon
Greenhorn
Posts: 17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ok, thank you!
What I didn't understand was the meaning of the word "program", now I see what is the goal of the specifications.


Marco
 
Ranch Hand
Posts: 105
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What I don't understand is how to write the lock, unlock and islocked methods and why they are needed. Also, I don't understand what will get mapped against the record number in the locking code. I have heard talk of cookies and Monkhouse's book just has me confused even more. Anyone have any examples?
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic