• Post Reply Bookmark Topic Watch Topic
  • New Topic

Files locked by JVM(?) on NTFS

 
L.A. Nilsson
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello All!

The environment:

Java 1.5, Windows Server 2003 / Windows XP (both using NTFS)
java.io.File

The situation

I process a lot of files of various sizes located in a folder. The processing involves reading only, no writing. When the processing is finished, I wish to move the files to other folders, depending on the outcome of the processing. Some files are to go to "success" and some to "error". I use someFile.renameTo( <pathToDestination> ) for this. Most of the time, this works excellent.

The problem

In about 5 - 10% of the cases it doesn't work. The file will not move. Those cases are weakly correlated to the size of the file. Most of the files I process are in the size of 2M - 8M, but some of them are much smaller, like 10kb - 300kb. I seems like the error never appear for the larger files, but in some cases for the smaller files. However, there seems to be no strong relation of the type "the smaller, the more likely to fail".

What I suspect / What I have done about it

It seems like somehow some kind of lock is attached to the file, thus causing the problem (alas, as you might guess, file systems are not my area of expertise). If I set up a thread to watch for files not being moved, looping and retrying to move the file every N seconds, the result is only the thread looping (apparently) for ever (in practice: till I hit Ctrl-C). So the unfortunate lock seems to be alive as long as the current JVM is alive.

What I wish from you / Why I don't solve the problem by more own hard work

I have tried to search for reports of this problem, but perhaps my lack of knowledge of Java/IO / NTFS terminology have caused me to fail.

I could do more experimenting and practical research, but I work under quite some pressure and perhaps some answer from you or some tip on how to best research this can save some valuable time for me. If so, I am most grateful!

Best Regards,
/L.A. Nilsson
[ October 17, 2005: Message edited by: L.A. Nilsson ]
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
After you read the file (& before you rename it), do you make sure to close() the FileReader or whatever class is accessing the file? Often this may get taken care of for you by garbage collection, but that's unreliable, and you can't guarantee when it will occur. (This could explain why you have no problem 90-95% of the time.) It is strongly recommended to close() IO classes in a finally {} block, to ensure that it happens even if an exception occurs.

Now if you're certain that you are closing things properly, and you still have this problem, then it's probably a bug of some sort that you will have to work around. (I remember occasionally seeing similar issues on NFS file systems; they're sometimes flaky.) On some evil systems, close() may return before the file is really, fully, completely closed. (It shouldn't, but hey, some systems are buggy.) You may need to insert a small delay (e.g. Thread.sleep()) in between closing the file and moving it. Or just move the file-moving code to a later point in the process, long after the close() was called.
 
L.A. Nilsson
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the answer, Jim!

Now, I am closing the stream for reading. Sorry for not stating that clear in my question. Inserting a delay doesn't seem to work, since it seems like the problem persists all the way until the JVM exits (at least as far as I can see, I have had a thread trying to move one of the files for as long as 20 minutes after the reading stream have been closed).

Right now I'm solving the problem by having another process start up when the first process is done and checking a report for files that is supposed to have been moved. This seems to work regardless of time elapsed between the two processes (another evidence pointing in the direction of the JVM somehow keeping the lock or failing to close one of the streams as long as it is alive).

For one and another reason I'm not very happy with this solution, and it will pose more problems for me later on I suspect, but maybe I'll have to stick to it ...
 
L.A. Nilsson
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm not well acquainted to java.nio. Would some magic in that library solve this problem?
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!