|
|
|
|
|
|
Win a copy of Five Lines of Code this week in the OO, Patterns, UML and Refactoring forum!
|
|
|
|
|
|
|
|
|
|
|
|
"I'm not back." - Bill Harding, Twister
// Releases the lock on a record. Cookie must be the cookie
// returned when the record was locked; otherwise throws SecurityException.
public void unlock(long recNo, long cookie)
throws SecurityException;
I just used java.math.Random and the nextLong() method to generate each key as needed.
How do you make sure that two clients do not receive a same cookie ?
I don't. If someone wants to use their cookie to try to unlock some other record that was locked by another client, well, there's a 1 in 2^64 chance that it will work. That's low enough for me. And consider - if someone is writing client code to try to unlock other clients' locked records, why not just use a cookie value of 0, or 1, or 124356789L, or just generate a random long of their own? You get the same 1 in 2^64 chance. Eliminating cookies that have already been used doesn't change this much, IMO.
but realistically, security isn't a big concern for this assignment
For the cookie, I didn't use a prime generator. For the most part, I don't think we're being tested on security issues. Just document your considerations and your design choices. I treated the cookie like a password; and passwords don't have to be unique or necessarily follow any other guidelines except size.
), I wrap the sequence back to LONG_MIN_VALUE.
)
Does that mean that we can do away with synchronized read and write operations (ignore),
Other operations, in particular those that take an explicit position, may proceed concurrently; whether they in fact do so is dependent upon the underlying implementation and is therefore unspecified.
). In fact, I am not sure about the correct interpretation to be given to that doc excerpt.and worry about only the concurrency on the server.
As my assignment states that "I can assume that only one program is accessing the database at any time". So the client checks the record lock in memory for update and delete operations, obtains it and does a transaction with the database. Hence no need to synchronize the read/write operations.
"I'm not back." - Bill Harding, Twister
When he finds a record he wants to unlock, all he needs to do is lock any other record, get the cookie, and then start trying cookie values, counting down from the value he just received. In comparison the 1 in 2^64 chance is much more secure.
Correctly-written non-malicious code should never get a SecurityException, I think. So for me, cookies are useless except as an extra test of correctness. And except for the fact that the API requires them.
Philippe is correct to be wary of FileChannel's thread safety, IMO
Phew !
Regards,
Phil.
Originally posted by Jim Yingst:
Greetings, everyone. Once more into the Philippe is correct to be wary of FileChannel's thread safety, IMO - the guarantees it offers are not absolute.
That is, if we do
"I'm not back." - Bill Harding, Twister
You may assume that at any moment, at most one program is accessing the database file; therefore your locking system only needs to be concerned with multiple concurrent clients of your server.
"I'm not back." - Bill Harding, Twister
Incidentally, since I used read(ByteBuffer) above - I also believe that FileChannel offers no guarantee that the ByteByffer will be filled, even if the file has more bytes.
Originally posted by Jim Yingst:
Howdy, Max.
Actually, you can trust the guarantees, but you have to be careful about reading too much into them.
I agree with this. When I said "the guarantees it offers are not absolute" I meant that the guarantees do not promise everything we might think or hope that they promise, and we must be careful with them. I do believe that the guarantees are obeyed (probably, excepting some possible bugs which will be fixed soon if not already).
For example, a write is atomic, if you don't move the position explicitly, and allow the FileChannel to write to it's next natural position. Jim, this is all on pages 284-285 of my book, so you should know this
I read this, but don't believe it offers the level of security you think it does. We need to provide random access to any record in the file, not just the next record. So either we have to use read/write methods that take an explicit position, or we set the position and then use a read/write that implicitly uses that position. You seem to advocate the latter approach. And it's true that the atomicity of the read/write is guarantted by the API. But what is not guaranteed is that there will be no interruption btween setting the position and performing the read write.
channel.position(100);
channel.read(buffer1);
while another thread does
channel.position(200);
channel.read(buffer2);
we may end up with something like
channel.position(100);
channel.position(200);
channel.read(buffer2);
channel.read(buffer1);
Here buffer2 gets what buffer1 was expecting, and buffer1 gets whatever is after the record at 200. This is no good; we'd need explicit synchronization to prevent interruption between the position() and read() methods.
Alternately, we can use the read/write methods that take an explicit position:
channel.read(buffer1, 100);
channel.read(buffer2, 200);
This works great as far as the reads are concerned. According to the API these methods may even process concurrently, but they're still guaranteed to each read from the appropriate place.
However, what if we have two thread doing:
channel.read(buffer1, 100);
channel.write(buffer2, 100);
According to the API these may also proceed concurrently. That's a potential problem. The read() will read from the correct position, but what it's reading may change underneath it. You may get a read that starts out with data from before the update, and ends with data from after the update. Unless, again, you protect your methods with synchronization.
I do believe that this type of dirty read is pretty unlikely, especially considering the record legnth is just 183 in my assignment. The system will probably finish each read/write atomically. And it's quite possible that on some particular implementations of FileChannel, this atomicity will always occur. However, it's not guaranteed by the FileChannel spec. So I advocate explicit synchronization if you want to ensure that dirty reads do not occur.
Incidentally, since I used read(ByteBuffer) above - I also believe that FileChannel offers no guarantee that the ByteByffer will be filled, even if the file has more bytes.
Originally posted by Bob Reeves:
Hi Jim:
In regard to your statement:
I also had this concern. FileChannel's read returns the number of bytes "actually" read. Unfortunately, this depends on what one's definition of actually, actually is. I noticed none of SUN's examples for FileChannle's read method enclose the call in a while loop, so I didn't use a while loop either. But, I'm not sure if looking at the implementation code is justified if the method contract doesn't guarantee the behavior.
I think SUN needs to put a Contract section in their Javadoc that explicitly states what behavior is guaranteed by a method. Fat chance?
Tx
"I'm not back." - Bill Harding, Twister
However, what if we have two thread doing:
channel.read(buffer1, 100);
channel.write(buffer2, 100);
According to the API these may also proceed concurrently. That's a potential problem. The read() will read from the correct position, but what it's reading may change underneath it. You may get a read that starts out with data from before the update, and ends with data from after the update. Unless, again, you protect your methods with synchronization.
I think there's a misunderstanding here. Reading changes the position of the of the FileChannel: thus, it's covered under the part of the documentation that says
Only one operation that involves the channel's position or can change its file's size may be in progress at any given time; attempts to initiate a second such operation while the first is still in progress will block until the first operation completes. Other operations, in particular those that take an explicit position, may proceed concurrently; whether they in fact do so is dependent upon the underlying implementation and is therefore unspecified.
Thus, the situation, as described, cannot occur.
public abstract int write(ByteBuffer src,
long position)
throws IOException
Writes a sequence of bytes to this channel from the given buffer, starting at the given file position.
This method works in the same manner as the write(ByteBuffer) method, except that bytes are written starting at the given file position rather than at the channel's current position.
This method does not modify this channel's position
Actually, it is in the documentation.
mean?Only one operation that involves the channel's position or can change its file's size may be in progress at any given time; attempts to initiate a second such operation while the first is still in progress will block until the first operation completes. Other operations, in particular those that take an explicit position, may proceed concurrently; whether they in fact do so is dependent upon the underlying implementation and is therefore unspecified.
means that you can use a FileChannel from multiple threads without worrying about disturbing its internal state unexpectedly. It doesn't not guarantee any resource that the FileChannel opeerates on.File channels are safe for use by multiple concurrent threads
Originally posted by Bob Reeves:
Hi Max:
This responds to your comment:
Respectfully, I disagree. My concern is that the method just might not return all the bytes requested in one read, whether another thread is involved or not. I've seen this behavior on Linux (in a single thread application) where a getBytes didn't return all the bytes known to be in the receive buffer. However, if I looped on a test byte count condition, then I obtained all the bytes.
So, I have this worry that if SUN says something like a method returns the number of bytes "actually" read, maybe the programmer should watch out.
Tx
Originally posted by Jim Yingst:
Hi, Max!
That's true for three out of four read methods recently surveyed. But you'll note that the code sample I provided (which Bob was responding to) uses
pos += channel.read(bytes, pos);
which does not change the FileChannel's position.
So, why not just use the implicit-position methods? Because they always rely on the assumption that the "current position" is the correct one, and that it hasn't just been reset by some other thread immediately beforehand. If multiple threads have access to a given FileChannel then the only way I see to prevent them from possibly interrupting each other like this is by using synchronization.
. Remember, dragging that FileChannel around with you, even as you're not using it, isn't a much prettier picture then opening it on a as-needed bases. The only argument I can see against it is that it's more efficient not to open/ close a connection,. My answer there is
And if you're going to sync critical operations anyway, then it really doesn't matter much if the operation was atomic or not, does it? So really, I'm at a loss as to why the FileChannel API was written they way it is; the guarantees it does offer are insufficient to be really useful, IMO. I'm perfectly happy doing synchronization myself - I just wish the spec had been clearer
.
Originally posted by Vlad Rabkin:
Hi Jim,
Hi Max,
Jim said:
So, I beleive Jim is right. The methods (write/read) having position in argument DO NOT CHANGE position in the FileChannel, so the CAN proceed concurrently!
Vlad
I don't see why read(ByteBuffer, long) (assuming that's what you meant) would need to block for anything. Could be missing something again... But there are a number of different people in this thread, and I know that some (like myself) aren't currently doing the just-in-time thing. The FileChannel question was first raised in this thread by S Bala, who seemed to be talking about multiple threads accessing a FileChannel, as are a number of the other people in this thread. If you want to talk about how FileChannel can be used safely in the contect of just-in-time FileChannel creation, that's fine - but I don't think this context has been clearly established in this thread for a lot of these comments. Many of the people reading this thread will not assume just-in-time creation unless it's explicitly stated. For those people, I say don't think of FileChannel's operations as "atomic" because while many are, some are not, and there are just enough holes in the system to screw you up if you're not careful. In contrast, syncronization is a simple and easy option, IMO:"I'm not back." - Bill Harding, Twister
The point here is that there is only one way to get unstable data, and that is to use the FileChannel.read(byte[],position) method in an environment that allows multiple threads to access the same FileChannel. As I recall, you're not doing this, correct?
Originally posted by Philippe Maquet:
Hi Max,
Does it mean that if I have only one thread which performs all writes (they should be blocking, right ?) and multiple threads which use the FileChannel.read(byte[],position), I am OK ?
Thanks,
Phil.
Originally posted by Jim Yingst:
[QB]Hello again, Max.
You'll notice that FileChannel.write(byte[],position) can change file size.
Good catch - I had overlooked that. So OK, among read/write methods only read(ByteBuffer, long) seems to be eligible to proceed concurrently. Which is still enough to create problems multiple threads access a FileChannel using that method, if some sort of write() is being performed on the section of the file being read. If the write is changing XXXXXXXXXX to OOOOOOOOOO, the read could theoretically see something like XXXOOOOOOO. It's fairly unlikely, and maybe impossible on many/all platform for all I know. But it's exceedingly annoying to me that the spec leaves this hole in its guarantees.
Reads, all of them, block too. However, the read(byte[],position) method only blocks for other reads.
Eh? Where did that come from?
The point here is that there is only one way to get unstable data, and that is to use the FileChannel.read(byte[],position) method in an environment that allows multiple threads to access the same FileChannel.
Well, there are other problems if you use a shared FileChannel using implicit-position methods, because there can be interruptions between setting the position and reading or writing. I gave an example of this earlier in this thread. This won't lead to reading XXXOOOOOOO when XXXXXXXXXX or OOOOOOOOOO is expected - but it might lead to reading YYYYYYYYYY (a different record entirely) instead.
Unless, of course, we use synchronization. Which is actually pretty simple. Or, unless you use separate FileChannels as you prefer.
As I recall, you're not doing this [sharing FileChannels], correct?
Who - Vlad? Dunno, I've lost track.
For those people, I say don't think of FileChannel's operations as "atomic" because while many are, some are not, and there are just enough holes in the system to screw you up if you're not careful.
In contrast, syncronization is a simple and easy option, IMO:
.
Note that I'm still putting those reads and writes in loops. That's because of the other problem with FileChannel, mentioned in several other threads above. Even if reads / writes are atomic, there's no actual guarantee (that I can find) that a read or write will actually use all the bytes we expect.
That is, if there's still space in a buffer, and the file still has unread bytes, a read() may nonetheless return prematurely, without filling the buffer. At least, the API seems to imply this, and offers no guarantee otherwise.
Now, I haven't been able to actually observe this problem in testing. Maybe it never actually comes up. I know it comes up with FileInputStreams fairly often, so I tend to assume it's possible here unless guaranteed otherwise. And I would argue that if partial reads/writes are possible, then they actually destroy the safety of just-in-time FileChannels. It doesn't matter if each read() or write() is atomic, if it's possible for the method to complete without delivering all the intended bytes. You can put the method in a loop as I do above - but without synchronization, there's no way to prevent interruption between loop iterations.
And even with synchronization, if you're using just-in-time FileChannels then each FileChannel is a separate instance. You'd need to sync on some shared shared monitor. A static variable or some such.
In contrast, if you have one FileChannel shared by many threads, you can simply use synchronization as I do above to guarantee that each read or write will be uninterrupted. Synchronization is not the complex beast often it's made out to be if you just use it. It's true that in some situations synchronization can be a performance bottleneck. But as has been frequently observed here, performance need not be a significant concern for this assignment.
I acknowledge that most of my concerns here are very unlikely to manifest as observable problems in this assignment. And if someone wants to bypass synchronization because they consider the risks to be acceptably low for their purposes, that's fine, as an informed decision.
But if people believe that there are actual guarantees that this problems cannot occur, that's what I disagree with.
Originally posted by Bob Reeves:
Hi All:
Not another post on the FileChannel! Sorry, but I want to add one detail.
Vlad's comment about whether writing three bytes to the start of a long file might not cause a block seems reasonable. Not to soapbox, but perhaps it's not good to microread text that's not professionally written. The javadoc for FileChannel says something like any operation that can change the file length will block. It doesn't say "any operation that changes the file" will block. Maybe this is the emphasis the author intended. The purpose might be to increase the performance of write operations by permitting concurrency.
Tx
Originally posted by Bob Reeves:
Hi All:
Not another post on the FileChannel! Sorry, but I want to add one detail.
Vlad's comment about whether writing three bytes to the start of a long file might not cause a block seems reasonable. Not to soapbox, but perhaps it's not good to microread text that's not professionally written. The javadoc for FileChannel says something like any operation that can change the file length will block. It doesn't say "any operation that changes the file" will block. Maybe this is the emphasis the author intended. The purpose might be to increase the performance of write operations by permitting concurrency.
Tx
Originally posted by Vlad Rabkin:
Hi Max,
Hi Jim,
[Max] You'll notice that FileChannel.write(byte[],position) can change file size.
Well, I thought about that 100 times, but what does it exactly mean???
Let's say write with FileChannel.write(bytebuffer, position), where bytebuffer has only 3 bytes on the beginning of the file (So, I am sure that this operation will not change the size of the file). Theoretically each write can change the size of file, but in practically this one NOT. So, is atomicy guaranteed or not???
Vlad
. The atomicy is guaranteed, because the spec says that any operation that may change file size is atomic. The compiler can't examine your code and figure out that you're not really going to be changing file size. When you ask for a writable FileChannel, and you call write on it, it blocks. That's what the language specifications say, anyway. And if you trust then language specs written by Sun, then Sun can't very well fault you if the specs go wrong.
But, the problem with Vlad point is that the compiler would have to examine your code, and know that you're only doing 3 bytes. Since it can't reasonably be expected to do this, then it has adhere to the specification. That doesn't mean that all methods always do so, but it's important to be clear that the method can't pick and choose when to adhere to spec and when not to.
Originally posted by Bob Reeves:
Hi Max:
This responds to your comment:
The motivation for by post was a decompile of FileChannelImpl, which is the type of the obect returned from RandomAccessFile's getChannel.
The decomile of FileChannelImpl shows no embedded JVM Monitor enter/JVM Monitor exit calls in any of the write methods. But all the position methods have them. Now FileChannelImpl calls IOUtil and FileDispatcher. Again neither of these contain Monitor calls in their write methods.
So, I think you can see why I conclude that the block on write must be in the native code. I'd also opinion that native code protects itself on block (ie. so that its file on disk information remains valid), not the user. Thus my conclusion.
Notice there is no customization of the compiled code for short buffers. Of course, the compiler couldn't do that! I think the native code looks at the buffer length, and branches according to file length. (Actually, the only time it must block is if the truncate method is called after the write. I'm guessing truncate isn't called frequently, so it might make sense to optimize this way.)
Tx
| Don't get me started about those stupid light bulbs. |
