Unreferenced interface should not be used.

[Gennady Shapiro]

read : http://java.sun.com/products/jdk/1.2/docs/guide/rmi/faq.html
There has been a lot of talk about using stale references for server clean up. Many people have mentioned Unreferenced interface as a way of doing this.
I dont think it's such a hot idea for 2 reasons.
1. There is a thing to remember : client crash timeout and transaction timeout are two different things. The Unreferenced interface was designed to detect crashed clients and thats why Sun sets RMI lease value (timeout) to 10 minutes -- an appropriate time period to identify a lost session and clean up session resources.
This time period should not be used as the transaction timeout simply because it's unacceptable to have a crashed client locking a record (or the database!!!) for 10 minutes! And this is exactly what happens when you rely on the Unreferenced to do locking cleanup. Of course, you can reduce lease value to 5 seconds or less (an appropriate time to identify a stalled transaction) but this will quickly take up your bandwidth as the number of connected clients grows. So if you use the Unreferenced interface you have to balance between a) Long Transaction Timeout and b) Huge network overhead.
2. There are 2 types of transaction locking : pessimistic and optimistic. Pessimistic type dictates the following algorithm:
lock()
read()
-- user thinking
modify()
unlock()
this would work fines if werent for user thinking. He might take 10 minutes to think, meanwhile the record remains locked. And that is unacceptable. The Unreference interface is really usefull only for this model if at all.
Now, the optimistic model dictates the following:
read()
--user thinking
lock()
read() (again to refresh)
modify()
unlock()
there is no user interaction within the transaction bounds. In normal program flow this transaction executes in under a second (2-3 max) therefore the record remains locked for about a second.
The Unreference is useless here. If may argue that if read() or modify() breaks the record stays locked forever. True, but if you do your exception hanling right you should be able to unlock the record as long as the connection is alive. If the connection phisically breaks between lock() and unlock() you end up with a stale lock, thats true. But again, waiting for 10 minutes to unlock it -- is hardly a solution.
I think that if you do have to do stale lock clean up you should create a Cleaner-Thread that removes locks older than say 10 seconds, that is an acceptable real-world transaction timeout.
I also think that stale lock removal is beyond the scope of this project and after loooong consideration I decided against doint it at all.(As long as I mention this in the documents)
Gentlemen, let me know what you think.
Thank you
[This message has been edited by Gennady Shapiro (edited October 30, 2001).]
[This message has been edited by Gennady Shapiro (edited October 30, 2001).]

[Richard Walter]

Gennady,
I had the same thoughts as you on this matter and agree with what you said. I have a TimerTask object that cleans up locks that are more than 10 seconds old. I still use unreferenced though as a secondary precaution which cleans up expired locks after 10 minutes. Not sure if this is overkill or necessary.
Rich

[Yuriy Fuksenko]

I agree. I think removing stale locks is beyond the scope of this project.
I could see 3 ways of doing it:
1. Using some time interval, it could be something on your own.
2. Making each client an RMI object, register it, and when client trying to lock record that already locked, check is previos owner alive.
3. Using Unreferenced.(I used this one)
1 and 3 require you to set some time interval (even though unreferenced give you default value). It does not make sense, if its value is not customizable, and it cannot be customizable because of assignment requirements.
From my experience with #3, I tested it with some amount of threads, each of them where locking all database, and than died. At this case, for same client, unreferenced method got called from a different thread, than lock, what happend, is unreferenced method executed before actual lock accured - so I ends up with synchronizing at my RemoteData class, so locks will be finished before releasing them with unreferenced.
Not sure what with number 2 - seems to be complicated and having perfomance problems, but if anyone tryed it - I would like to hear it.

[Yuriy Fuksenko]

I agree. I think removing stale locks is beyond the scope of this project.
I could see 3 ways of doing it:
1. Using some time interval, it could be something on your own.
2. Making each client an RMI object, register it, and when client trying to lock record that already locked, check is previos owner alive.
3. Using Unreferenced.(I used this one, and I used 2nd locking model)
1 and 3 require you to set some time interval (even though unreferenced give you default value). It does not make sense, if its value is not customizable, and it cannot be customizable because of assignment requirements.
From my experience with #3, I tested it with some amount of threads, each of them where locking all database, and than died. At this case, for same client, unreferenced method got called from a different thread, than lock, what happend, is unreferenced method executed before actual lock accured - so I ends up with synchronizing at my RemoteData class, so locks will be finished before releasing them with unreferenced.
Not sure what with number 2 - seems to be complicated and having perfomance problems, but if anyone tryed it - I would like to hear it.

[Peter den Haan]

Originally posted by Gennady Shapiro:
1. There is a thing to remember : client crash timeout and transaction timeout are two different things.

Obviously. I don't think anyone has proposed using Unreferenced for anything other than client crash cleanup. There is no requirement or provision in the FBN project for a transaction timeout - you would have to amend the API.

2. There are 2 types of transaction locking : pessimistic and optimistic. [...] The Unreference is useless [in optimistic locking]

Your description of optimistic locking is inaccurate - what you are describing is still pessimistic locking, albeit for a brief period of time, and no locking otherwise. But that's not the point. The Unreferenced option is still as useful as ever in the second scenario (which I hope everyone is using, the first one is madness!) because, as stated, it is not a transaction timeout. It is simply a way to clean up locks if a client crashes (or its power goes or the network or...). And mishaps can happen while a client is holding a lock irrespective of how you organise your transactions.

If the connection phisically breaks between lock() and unlock() you end up with a stale lock, thats true. But again, waiting for 10 minutes to unlock it -- is hardly a solution.

Go tell that to Oracle, Microsoft and the other big guys out there. No database I know will aggressively time out transactions after a few seconds just like that. A couple of times I've been in a situation where it would take the (Oracle 8.1.5) database virtual eternity to clean up locks after a bad client crash. It's bad when that happens. But the alternative would make the database unworkable for applications that needed lengthy transactions.

I also think that stale lock removal is beyond the scope of this project and after loooong consideration I decided against doint it at all.(As long as I mention this in the documents)

Here we agree. In fact I've stated repeatedly that I only did it because it was so laughably easy to do; decent and well-defined behaviour in case of client crashes at the cost of just three lines of code, features don't often come so cheap and it sure beats having no cleanup at all.
- Peter

[This message has been edited by Peter den Haan (edited November 01, 2001).]

[Gennady Shapiro]

Originally posted by Peter den Haan:

Originally posted by Gennady Shapiro:
[b]1. There is a thing to remember : client crash timeout and transaction timeout are two different things.Obviously. I don't think anyone has proposed using Unreferenced for anything other than client crash cleanup. There is no requirement or provision in the FBN project for a transaction timeout - you would have to amend the API.

Well, what is client crash cleanup? If it's not the locks that the client imposed, what is? You may say it's the session object -- here I agree, it should be cleaned up. But this will happen anyway -- dgc will garbage collect all unreferenced objects even without your interaction. I consider the Unreferenced a remote version of finalize. and finalize is designed to do "resource clean-ups" like files, network connections maintained by the session objects and should be explicitely closed. Also, I do recall that Sun's guideline is not to nullify objects but let the gc to collect them at its own discretion. Therefore the Unreferenced should not garbage-collect session objects either. I still do not see how the Unreferenced interface is usefull in any way.

2. There are 2 types of transaction locking : pessimistic and optimistic. [...] The Unreference is useless [in optimistic locking]Your description of optimistic locking is inaccurate - what you are describing is still pessimistic locking, albeit for a brief period of time, and no locking otherwise. But that's not the point. The Unreferenced option is still as useful as ever in the second scenario (which I hope everyone is using, the first one is madness!) because, as stated, it is not a transaction timeout. It is simply a way to clean up locks if a client crashes (or its power goes or the network or...). And mishaps can happen while a client is holding a lock irrespective of how you organise your transactions.

My description of optimistic model may be simplistic and does not reflect record state check,conditional updates, etc. But for the sake of this discussion it is accurate enough since it shows (1)read (2) re-read (3) update. You are correct its not the point. What important is that Unreferenced gives you a session time-out notification , not the transaction time out. By saying that the Unreferenced should be used to clean up locks you effectively saying "if client crashes lets wait until his session times out then we'll clean up locks." I dont think it's a great idea. All app servers differentiate between object time to live and a transaction-time out.

If the connection phisically breaks between lock() and unlock() you end up with a stale lock, thats true. But again, waiting for 10 minutes to unlock it -- is hardly a solution.Go tell that to Oracle, Microsoft and the other big guys out there. No database I know will aggressively time out transactions after a few seconds just like that. A couple of times I've been in a situation where it would take the (Oracle 8.1.5) database virtual eternity to clean up locks after a bad client crash. It's bad when that happens. But the alternative would make the database unworkable for applications that needed lengthy transactions.

You are right , the DBMS will not aggresively time out the transactions. But they should, if not them then the App servers and/or Distributed Transaction Managers like Weblogic do, and a system designer should always set maximum reasonal transaction time out in the descriptor. Going with the default behavior is not always a good idea. In this particular case 5-10 seconds is more than enough for the transaction and locks should not be imposed for longer time. Thats the good design in my opinion.
I appreciate you all taking time to discuss this. This probably in no way will affect your score, just interesting to see points of view.

[This message has been edited by Gennady Shapiro (edited November 01, 2001).]

[Peter den Haan]

Originally posted by Gennady Shapiro:
Well, what is client crash cleanup? If it's not the locks that the client imposed, what is?

The locks are the most important. What I meant to say (but probably didn't) is that the intent is completely different from your short-lived transaction timeout idea. You appeared to be attacking Unreferenced for not being something it was never meant to be.

I consider the Unreferenced a remote version of finalize. and finalize is designed to do "resource clean-ups" like files, network connections maintained by the session objects and should be explicitely closed.

That's correct (although it comes with more guarantees than finalize() does). And that's exactly how it's being used - as a last opportunity to clean up resources if everything else failed. In this particular application, the resources in question are record and database locks.

for the sake of this discussion [my description of optimistic model] is accurate enough

It's a terminology issue. To avoid sidetracking the discussion I'll put my idea of optimistic locking in a footnote below.

By saying that the Unreferenced should be used to clean up locks you effectively saying "if client crashes lets wait until his session times out then we'll clean up locks." I dont think it's a great idea. All app servers differentiate between object time to live and a transaction-time out.

Ah. But this is not an application server. It's a database. We seem to agree that "real world" databases do not time out transactions, except (ahem) when a client crashes. And they take their own good time doing that. This is exactly the behaviour that the use of Unreferenced gives you. Not only that, it's exactly the kind of thing that Unreferenced is intended for. We don't seem to disagree that, if this is what you want to achieve, Unreferenced is the #1 way to do it.
What we do disagree on is whether this is what you want to do in the first place. Me, I was a database guy well before I got into Java, and I can see why databases work that way. In general, a database transaction cannot be timed out after a few seconds or even a few minutes (unless you make it configurable) because there are applications where you legitimately need such transactions. I also see why application server environments impose their own, different constraints.
What this means is that ultimately, it's just another design choice that you have to make and document with your submission. Are you trying to implement a general-purpose database, or a specialised one for the flight-booking system and other applications that require short-lived transactions? Since Sun explicitly states that you are to design for re-use, I chose to work towards a general-purpose DBMS as much as possible within the scope of the project (possibility to have multiple tables, no constraints on lock duration). I would contend that this is a legitimate choice (but certainly not the only one). I would also contend that providing Unreferenced-based cleanup is far preferable to having none at all, no matter what your design choices are.
Thank you. I love discussions like this. They, and the thought and learning processes that are behind them, are a big part of what makes the SCJD certification a valuable asset.
- Peter
PS. Optimistic locking means you take a snapshot of a record at the start of a long transaction and modify it under the assumption that it won't be modified by someone else behind your back. If, by the time you want to save your changes, the record turns out to have been modified by another process, the transaction fails. Optimistic locking is usually implemented by adding a versioning field to the table.
The FBN application does none of that. It is perfectly OK for a record to be updated by another client between reading it for the first time and allocating seats. This means there is neither a pessimistic nor an optimistic lock; there is simply no lock at all. That is why you have to refresh (after setting a pessimistic lock) to ensure that you don't work with stale data.
[This message has been edited by Peter den Haan (edited November 02, 2001).]

[Gennady Shapiro]

Peter,
OK. Your point well taken. According to the specificaion, FBN is a client server. That means RMI server is not a middle-tier server but a part of the database. I was attempting to cleanly separate layers : DB-Server-Client. Nevertheless...
I see that we disagree at one thing:
your point is "the database has to clean locks (stalled transactions) when it detects that a client crashes."
my point is "the database should have an internal transaction time-out that cleans locks regardless of the client's state."
I think this is pretty acurate description of the problem in hand.
I am not a DB expert but I have to believe that a DB has to allow you to set MAX_TRANSACTION time that is used to roll back transactions if this time has been exceeded. I know App servers do that , and that is a big deal in (distributed) transaction management, I think that this functionality is modeled after databases so there should be a way to limit FBN lock time-to-live without violating the grand-DB-transaction-scheme. The implementation is simple (and widely discussed here):
have a lock monitor thread that rolls back stalled locks using that MAX_TRANSACTION time.
Now, if you have to rely on client crash notification you have to rely on the Unreferenced. But I am a big proponent of using this as the last resort.
Also,
I think that the optmistic locking fits very well into FBN if you consider the Number od Available Seats the marker field.
The algorithm is this.
1. read the record
2. user is thinking
3. re-read the record
4. has record been changed?
5. ? fail : update.
or even a better modification of that
4. is there still enough seats?
Thanks
[This message has been edited by Gennady Shapiro (edited November 02, 2001).]

[Peter den Haan]

Gennady,
The main discussion appears to have reached a nice point of closure; different goals, different implementations. Certainly if I were to code an application server layer on top of my database I would consider aggressively timing out transactions.

Originally posted by Gennady Shapiro:
I think that the optmistic locking fits very well into FBN if you consider the Number od Available Seats the marker field.
The algorithm is this. [...] or even a better modification of that
4. is there still enough seats?

It's that last modification that changes the "optimistic lock" into "no lock". The term "lock", in the meaning I'm familiar with, means that no-one else can/should modify the record you've locked. A pessimistic locking scheme takes the pessimistic view that there will be attempts to modify the record and makes such modifications impossible. An optimistic locking scheme takes the optimistic view that there will not be any attempts to modify the record and is content to merely detect that the unthinkable has happened. The FBN application, as most implement it (the modified way you've outlined above), simply doesn't care if there were any modifications between the initial read and the seat allocation - there's no locking semantics at all. There's merely a business rule that after modification the #seats should be >= 0.
- Peter

[Gennady Shapiro]

I love computer scince. Isn't it wonderful how a little tweak of specification/requirememnts solves most every problem?
So I guess using the Unreferenced interface is up to the developer as long as he clear on his intentions.
just a note on Optimistic locking.
I omitted the locking notation for simplicity. The real algorithm would look something like this:
1. read the record
2. user is thinking
3. lock()
4. re-read the record
5. has record been changed? (or is there still seat available?)
6. ? fail : update.
7. unlock()
now we have locking semantics and everything looks hunky dori.
P.S. I could never understand why the specification forces the client to perform locking logic...maybe because it's the easiest way to do transaction demarkation (in this case), but really not the brightest.