4.1) A scalable server must use thread-pooling to avoid creating an excessive amount of threads,
ultimately resulting in a denial-of-service.
4.3) 4.2) can be avoided by having 2 threadpools, one for constructing clientrequests and one for executing queueries. Two pools are better than one since it maximizes availability of the server to clients (this assumes that, in general, constructing a clientrequest is cheaper than carrying out a query)
4.4) Requests created by the first pool is enqueued on a FIFO-queue, which is dequeued by the second pool running the queries. This maintains a degree of fairness in the order of execution. And of course a queue is necessary if we cannot have an unlimited amount of threads for running queueries (refer to 4.1)).
4.5) The server should use callbacks to avoid congestion in the traffic going into the server.
I have a prototype for the above that works, however, it relies on RMI for constructing clientrequests and enqueueing them.
Using a threadpool combined with a FIFO-queue and doing callbacks ARE standard/well-known solutions for achieving SCALABILITY as far as I know.
You mention that RMI uses a threadpool, perhaps you could tell me a bit more about how it works? Is the threadpool part of RMI or actually just the VM re-using threads?
Is this threadpool self-adapting to the load somehow?
Originally posted by Alexander Gunnerhell:
No, I won't implement a full-featured DBMS since there's nothing in the assignment that indicates or states that this is a current or future need/requirement.
However, the requirement for scalability is strongly hinted in the first sentence describing FBN.
Aside from RMI I have also researched thread reuse in the JVM in general, I haven't found one single trace of that current SUN JVMs have this feature. So the db threadpool stays.
Dequeueing and execution of the clientrequests in the "database" will be managed by my own threadpool.
These threads will also handle the callbacks to the clients.
The reason for having multiple threads for executing clientrequests in the "database" is that a thread in the db will typically be blocked for most of it's scheduled time since it's doing I/O.
The reason for managing those multiple threads in a pool is too be able to limit the amount of threads for executing clientrequests to avoid the overhead of too many threads.
You're correct about the callbacks. In the prototype the client observes a remotely observable server-object. When that observable server object changes state (i.e. a clientrequest has finished) it executes a method on the remotely exported observer object in the client.
Comparing my "server" with a J2EE application server is very exaggerated, but I guess you were joking about that part as well as the part about paying my re-submission.
Tell me when you're ready to pay
Even if he thinks too many advanced mechanisms is necessary to scale the database and I have to change approach, I'm still very glad I've done all this thinking/design as well as the prototype.
Do you know a good tool for finding "hotspots" in an application? Preferably I'd like a tool that I can use to instrument my code and then have it report how much time is spent in different parts of the code.
I believe the main problem will be that searches will require traversing the hashtable and set readlocks for each and every element included in the search.