(NX Contractors): should RandomAccessFile variable be a member of Data?

[Rob Zidsen]

Right now I have a File as a member of my Data class and in all my methods I create a new instance of RandomAccessFile on the File. It's a bit more convoluted to code because I have to open and close RandomAccessFile all the time. Should I change it?

[Andrew Monkhouse]

Hi Rob,
The questions I think you need to be asking yourself are:
Do you have the instruction "A clear design, such as will be readily understood by junior programmers, will be preferred to a complex one, even if the complex one is a little more efficient."? If so, do you feel that you meet this requirement, or should you refactor to make it simpler to understand?
What is your reasoning behind opening and closing the file all the time? There may be good reasons for it (Max has come up with some) but do you need it for your design?
Regards, Andrew

[Rob Zidsen]

Hi Andrew,
If there is only one RandomAccessFile that all threads are sharing, couldn't this lead to synchronization problems? For example:
Thread one calls seek(position) just before it does a write. After the seek but before the write, another thread interrupts and grabs priority. This second thread then calls seek(anotherPosition) and does a write operation. When the original thread gets priority back, can it potentially write to the wrong location?
This is the reason I instantiated local RandomAccessFiles everywhere. to prevent something like that from happening. Is such a scenario plausible?

[Niall ORiordan]

Just a little bit of advice for you.
Why are you making all changes directly to the file?
Can you not think of a better way of making changes and storing the database?
File access is extremely slow and also very hard to make threadsafe..
Maybe you should reconsider your design from scratch?
Cheers,
/Niall

[Rob Zidsen]

what would you suggest? store it in memory? what if the file was very large?

[Jim Yingst]

For what it's worth, I think a lot of people here make changes to the file immediately. Most also read from the file every time a read() or find() is performed. Personally I keep everything in memory so that read() and find() don't need to bother the file - but on any update(), create() or delete() I also write the changes to the file right away. Otherwise you need to consider how and when will the updates be made to the file? There are certainly ways to do this, but there are possible complications too. I think it's a perfectly viable solution to read and write to the file as necessary.
File access is extremely slow and also very hard to make threadsafe..
FileChannel makes it much faster. There are several approaches to thread safety, which need not be very difficult at all really. And really, if you're keeping data in memory, shared by multiple threads, then you need to synchronize access to that memory. Which is no more or less difficult than synchronizing access to the file, I think. The only difference is that if the file access is slow, the synchronization may cause that slowness to affect other methods as well. But file access with FileChannel is not that slow, and efficiency isn't a big requirement for the assignment anyway.

[Rob Zidsen]

I'm having this really strange problem with file channel now, and I hope someone can help me.
basically I have a method I call that checks the validity of a record. I use this method in my read and modify methods. I was using RandomAccessFile, and everything was fine. Then I just changed to FileChannel because everyone's saying how much more efficient it is, and it's messing up my db when i perform updates, deletes, etc.
here's the part using random access file:

//get the value of the validity flag
            rafDataFile.seek(offset);
            short validFlag = rafDataFile.readShort();

and here's the part using file channel:

ByteBuffer validFlagBuf = ByteBuffer.allocate(2);                        
fcDataFile.read(validFlagBuf,offset);
short validFlag = validFlagBuf.getShort(0);            

I really don't understand why such a small changed messes everything up.
In all my modifying methods, I use FileChannel as well. I'm using the same FileChannel instance in all my calls, since I've made it a member of Data. Is this somehow causing the problem?

[Jim Yingst]

Your new method should read bytes the exact same way the old one did, but it leaves the file in a different state. What the original method did was:

Set file position to offset.

Read two bytes.

File position is now offset + 2.

This position is significant if there are other subsequent methods that read or write immedately afterwards - they may simply use the previous position without resetting it themselves.
Your new method, instead, simply reads two bytes from offset and offset + 1. It does not actually change the file position at all. So any other methods that use that file position are now broken.
Here's a simple alternative:

ByteBuffer validFlagBuf = ByteBuffer.allocate(2);                        
fcDataFile.position(offset);
fcDataFile.read(validFlagBuf);
short validFlag = validFlagBuf.getShort(0); 

The postion() method explicitly sets the file position, and the read(ByteBuffer) implicitly increments the position for each byte read. Just like RandomAccessFile did.
Note that the file position of a FileChannel and the file postion of the associated RandomAccessFile both refer to the same thing. If you're using an RAF and FC simultaneously (and you got the FC from the RAF using the getChannel() method) then changes to one object affect the other. So you can replace RAF method with FileChannel methods, one at a time, and if you do it right any RAF method that rely on the position being set correctly, will also work if you set the FileChannel's position instead. As I did above.
[ August 13, 2003: Message edited by: Jim Yingst ]