URLyBird: DB with 'deferred write' capabilities

[Anonymous]

Hi all,
I guess you'll see me around for a couple of weeks. I downloaded the assignment yesterday and have been browsing through this board for the last couple of hours...
The first thing when I think about file-based databases (well, in fact, even SQL DBs are...) is "deferred write". How is the DB generally implemented?
If I did implement deferred write capabilities, I would load all records into a data structure upon start up of the application and assign self-made rec numbers. Each alteration of the data would alter it in the memory and start an individual thread that would do the same changes on the physical file. Performance-wise this sounds like a great idea, but numerous issues arise such as increased complexity for exception handling (pass I/O errors in thread back to GUI!?).
A "deferred write light" would no alter the physical file after each update but rather use some sort of check point interval. A separate thread could for example alter the file in a 60 second interval or so - and of course when the application is terminated. Then again what happens if the app crashes right before the thread runs -> several modifications might be lost.
If no deferred write capabilities are implemented, each alteration of a record takes some time because each invocation of a method that modifies the record, results in I/O operations.
I know from discussions I've read that performance doesn't really count, but still... What's your opinion on this? Go for the KISS approach (assuming that my first approach is not considered KISS)?
Regards,
Marcel

[Andrew Monkhouse]

Hi Marcel,
Welcome to JavaRanch.
As you have already noted, dealing with performance issues is not a requirement for this assignment. However there are quite a few members who are developing read caches to improve performance.
The biggest issue I would have with a write cache or a deferred write capability is - what happens if you cannot perform the write at the later time? You have already indicated to the user that the booking has been aproved.
If you do decide to write at the time the user does the booking, you are only writing a very small amount of data, so this is not likely to cause you a performance problem.
Regards, Andrew

[Anonymous]

Hi Andrew,
Thanks for your comments.

However there are quite a few members who are developing read caches to improve performance.

How is a read cache different from deferred write? What are its implications?

If you do decide to write at the time the user does the booking, you are only writing a very small amount of data, so this is not likely to cause you a performance problem.

You're probably right. But still, at that very moment all other threads that would like to have access to the data - read or write - have to wait until the semaphore has been released. If I got that right (looking at it from the users perspective), the GUI freezes until any alteration to the data the user submitted has been written into the file.
Of course any alteration could still be carried out through an individual thread, but then the same problems (how to report back errors, successful booking has already been indicated) arise.
Regards,
Marcel

[Andrew Monkhouse]

Hi Marcel,
Let me start by saying that I am not trying to push for a read cache. I think it is a nice idea, but I don't believe it is essential. However I am willing to continue discussing it since you are concerned about performance.

How is a read cache different from deferred write? What are its implications?

A read cache stores a copy of the data in memory. Whenever a client wants to read a record, the data is taken from the memory cache. This will give you a performance benefit for the most common operation (reads on the database), and will ensure that reads can still occur while the physical file is being updated (you will still need to ensure that a user cannot read the individual record being updated though .
However a read cache implies that writes are still immediate. If a client successfully calls Data's update() method, the client can be guaranteed that the data has been persisted.

You're probably right. But still, at that very moment all other threads that would like to have access to the data - read or write - have to wait until the semaphore has been released.

That is where a read cache will give you a benefit - the reads can continue even when writes take place.
As I mentioned before, the actual writing of the data is so small that even without the read cache this would not be a real problem. The writing to disk should complete so fast that the user will not even notice the delay. You would need to have a huge amount of people updating records simultaneously for the write cache (deferred writes) to give a noticable performance gain.

If I got that right (looking at it from the users perspective), the GUI freezes until any alteration to the data the user submitted has been written into the file.
Of course any alteration could still be carried out through an individual thread, but then the same problems (how to report back errors, successful booking has already been indicated) arise.

I think any client operation (not just update operations) should be run in an individual thread. There are a number of steps that must occur betwen when the user performs some action in the GUI and when they get the results displayed to them. There is processing on the client, processing on the server, network operations, and database operations. All of which can cause a noticable delay. So you should consider using a separate thread to do these operations, that way the GUI can provide usefull information to the client so that they don't believe it has frozen or hung.
Yes, you will have to think about how to get notification from the worker thread back to the GUI thread (perhaps the Observer pattern will help you out), but you will have a much more polished solution.
Regards, Andrew

[Anonymous]

Hi Andrew,

I think any client operation (not just update operations) should be run in an individual thread

Are you really certain about this? I see your point with the Observer. It is indeed the one pattern I would use to achieve such functionality.
But at what price this this threading come?
It is nice that the GUI is responsive again right after an action (e.g. book room/flight) has been submitted. However, the GUI can only display not-so-relevant (don't wanna say non-relevant ;-)) information until the thread has finished. The table can only be updated once the action has been processed. So, either the table shows outdated data while the action is been processed or it shows no data at all because it's been emptied before the thread starts.
This sounds like I want to ease my way out of this - and maybe I do - but it really looks to me as if the benefit for the user was (if at all) only minor. Maybe my mind just isn't broad enough to see all the benefits.
Regards,
Marcel, Switzerland

[Andrew Monkhouse]

Hi Marcel,
I am not suggesting that you allow the users to do anything while waiting for a response from the server - all you may want to do is pop up a dialog telling them that the action is being completed (possibly with some component which gets updated so that they can see that the program is still working. In best case scenario, such a dialog would only flash on the screen. But the point is that the screen will still repaint / behave normally while the server is busy.
The problem with the alternative is that while the user is waiting for the comand to complete, the gui becomes unresponsive, and may not repaint correctly (for example if it gets obscured by another window, when you bring it to the front again it may not repaint the obscured section) - so a user may feel that the client has crashed.
But if it makes you feel any better, lots of people have passed with very good scores without creating this separate thread.
Regards, Andrew

[Anonymous]

Hello Andrew,
Thanks for yet another super reply - and your patience with me. I see your point very clearly now and I think I'll go for the threaded solution...
Regards,
Marcel