You HSQLDB developers probably already know this... so I'm just explaining to others...
In browsing the server start up code it seems that a new thread is created and started whenever a connection to the server is opened.
Due to this the listening thread validates / configures the incoming connection before creating a new thread to handle the connection (the stuff that goes in the beginning of Server.handleConnection() ). This means that the listening thread spends that validation / configuration time away from the accept() method.
As a consequence the risk of the server dropping connections under heavy load is larger, than if the listening thread handled over the incoming socket to a handling thread instead. The listening thread would then return immediately to the serverSocket.accept(); Of course the drawback of this approach is that you risk creating a thread for an invalid connection. What a waste!
One possible solution to this "problem" would be a thread pool. The threads handling the connection would already be ready and waiting. Thus no wasted threads are created. A thread handling an invalid connection will just close it, and return to idle.
A thread pool may also solve some of the OutOfMemoryException some developers seem to be getting. A thread may take a lot of system resources (up to 2 MB on some systems). Under heavy load the HSQLDB server will create unlimited threads, all running at the same time, resulting in quite a lot of system resources being allocated at a time.
I've once tied up a Sun server with 2 GB mem by running a 1.000 threads at the same time... My application did take around 2 GB of mem... and the threads were hardly even running. The server was swapping like hell :-/ Later I learned about the amount of resources a thread takes from the system, and put an upper limit to the amount of threads running.
But what are your plans in this area?
The drawback o
Currently there are no plans to change this. If someone volunteers to rewrite the code, they are welcome. It would be a good idea to limit the number of allowed concurrent connections to the server.
If this code was rewritten there should be a difference
between the number of handling threads in the thread pool, and the number of queue connections in the pool.
That way you could set a maximum of 5-20 threads handling the connections, and perhapsa max of 20-40 connections in the queue waiting. The handling threads will just take connections out of the queue when ready to handle it. That way 5 connections can handle more than 5 connections. Connections will get queued until a thread is ready to handle it.
By the way, did any of you take a look at the minimal changes needed in the org.hsqldb.Server class to make it run with the org.hsqldb.concurrent.ThreadPool I uploaded to the patch tracker along with the connection pool stuff?
Just a few changes in:
Server - has member Thread Pool added.
Server - handleConnection calls the new method
instead of starting a new thread for it.
That's about it, I think.
New issues come up all the time and I haven't had the time yet to check the code. As I suggested in the tracker, we shall review it over a few weeks.
I think if your client is operating with a proper JDBC connection pool then this feature really isn't worth it.
You might be right about that, Elias. Especially considering that we are working on a connection pool to ship with HSQLDB. If people use that connection pool, thread pooling will be a lot less important.