Concurrency problem

[Christiaan Thamm]

Hi guys

Im having this problem in my data access class where im trying to read record from the database file. I force 100s of threads to simultaneously look for record. O and in my design n read my records from the db directly

What happens, as far as i can tell, is that in my read method i specify the location of the record and use file.seek(location) to go to that location. when i want to start reading the record i find that the file pointer has moved away from what was supposed to be the location at file.seek(location). This whole method is locked by a reentrant writelock. I checked and nowhere where i access the file do i not have this lock

Im not sure what is happening here. Is it possible that between me calling file.seek and reading the record the jvm decided to give access to another thread, which then moved my file pointer somewhere else? I thought the locking was supposed to prevent this, im not sure

Thanks

[Roel De Nijs]

Reading a random access file is not an atomic operation. You have to perform 2 actions: position file pointer and read the record length. So when having multiple threads that should be made thread-safe. You have many options to do so: using synchronized block, marking method as synchronized, using new concurrency api,...

To pinpoint the problem I would mark your complete read method as synchronized. That means no other thread can access this method as long as another one is executing the code of this method. If your problem is solved using this approach, it will definitely have something to do with your ReentrantLock. If the problem is still present when using synchronized methods, we'll need to look at other possible reasons.

[Christiaan Thamm]

Roel De Nijs wrote:Reading a random access file is not an atomic operation. You have to perform 2 actions: position file pointer and read the record length. So when having multiple threads that should be made thread-safe. You have many options to do so: using synchronized block, marking method as synchronized, using new concurrency api,...

To pinpoint the problem I would mark your complete read method as synchronized. That means no other thread can access this method as long as another one is executing the code of this method. If your problem is solved using this approach, it will definitely have something to do with your ReentrantLock. If the problem is still present when using synchronized methods, we'll need to look at other possible reasons.

Ill try synchronization out to see what happens, but just as an example have a look at this method piece:

recordsLock.writeLock().lock();
        try {
			file.seek(locationInFile);
			System.out.println(Thread.currentThread() + " inside lock of retrieveContractor with location " + file.getFilePointer() + " LocationinFile = " + locationInFile);
			file.readFully(input);
		} finally {
			recordsLock.writeLock().unlock();
		}

I put a printline in there to see what happens. Sometimes my locationinfile variable is different to what the getFilePointer pulls from the file... This obviously breaks my read becuase i asked it to read at location 70 but its actually sitting on location 5222

[Roel De Nijs]

Maybe each thread using its own data access class (and thus also its own recordsLock)?

[Christiaan Thamm]

Nope the threads all share the data class plus my recordlock is static. ugh why o why

[Christiaan Thamm]

synchronized (file) {
			file.seek(locationInFile);
			System.out.println(Thread.currentThread() + " inside lock of " +
					"retrieveContractor with location " + file.getFilePointer
					() + " LocationinFile = " + locationInFile);
			file.readFully(input);
		}

So i put a breakpoint at the printline to check when my locationInFile and file.getFilePointer() is different. When it hit i had a look at the threads. The current thread picked up the differrence. a different thread that was in "MONITOR" status has already written the printline statement above. This thread has its locationInFile at the point that my current thread picks up. Is it possible that thread 1 called file.seek and stopped. Thread 2 then enters the code and calls file.seek with its own location and then blocks. Thread 1 then continues with a wrong pointer location?

This is driving me crazy. ive been looking at this for hours. please help. Ive had synchronise and reentrant locks now im having the same problem.

[Roel De Nijs]

Did you try synchronizing the complete method? Do you still have this issue, because then it's not an issue with the code in this method, but is the problem elsewhere.

[Himai Minh]

Christiaan,
Do you mean this scenario?
1. Thread 1 seeks at location 70 and starts at location 70 , read the next 184 bytes.
2. Thread 2 is supposed to seek at location 170 for example , but it ends up seeking from (70+184) location
3. That means thread 2 cannot move to 170 after thread 1 finishes reading from location 70.

This happens frequently with a large number of threads, eg 100 threads?

[Himai Minh]

Hello Christiaan again.

If my previous post describes your issue, you may want to try this:

//Make sure your readFile method is access only by 1 thread at a time.
public synchronized readFile( File file ){
       file.seek(....);
       file.read(....);
       
}

[Roel De Nijs]

Himai Minh wrote:If my previous post describes your issue, you may want to try this:

In what is your code different with the code he posted in his last reply? Exactly the same. And by using synchronized you just try to make your code thread-safe, so why would you have to worry about making sure the readFile-method is accessed by 1 thread at a time He tries to develop a multi-threaded application, where 10 threads should be able to read different records concurrently.

[Himai Minh]

I read Christiaan's description but I am not sure if I understand his description correctly.
To my understanding, Christiaan's original code is like this:

//pseudo code: 
  public void  read(){

          //some implementation here

          // a synchronized block here
          synchronize(file){
                //read the file here....
          }
 
        //some other  implementation here.....
  }

To my interpretation, Christiann's original read method is not synchronized, and it only synchronizes a block of the code.

[Christiaan Thamm]

I Synchronise the block that accesses the file (move pointer and read), not the whole method. I didnt think it should be necessary to synchronise the sections where i handle the data that i read...

[Roel De Nijs]

Yes, that's true. In normal circumstances having a synchronized block around the seek and read methods are sufficient. But it's not working at this moment, so we are investigating what's causing this issue. So to limit the possible issues, you mark the complete read-method as synchronized, you give it another go and see what happens.

[Christiaan Thamm]

Debugging thread problems is quite difficult becuase sometimes my problem doesnt occur. Its possible that its fixed now, unless i was lucky that it worked a few times in a row. My question is does calling the length method on RandomAccessFile move the file pointer somehow? A method that called length wasnt synchronised, and doing this might have fixed it? Cant find anything on a google search though... Lets hope that as soon as I commit this post my debug doesnt crash

[Roel De Nijs]

Based on this bug report and this thread that could be the cause of your issues. Good job!

[Christiaan Thamm]

Roel De Nijs wrote:Based on this bug report and this thread that could be the cause of your issues. Good job!

Geez okay thanks for your help as always Roel, much appreciated. Glad I found this was driving me crazy