I think that all of this can wait. But. Since it sounds like some of
you find this to be a must have immediately, here's my suggestion.
Here is my refined proposal for load-balancing/fail-over. It is
derived from the JServe system. It's clean, lean and reliable.
First of all, the load balancing logic is in the Adaptor/webserver
module.
The key is changing the address.text file so it can have multiple
entries.
Part of the AppServer configuration file is the port to run on. We
change the Adaptor logic to consider that a starting port number. When
an AppServer starts, it will read the address.text file and see if
there are any other entries there, if so, it starts at the next higher
port and adds an entry to the file, otherwise it starts at the base
address and adds that entry to the file. (note, there is another
verion of this below, related to starting the AppServers and writing
to address.text)
When an adaptor starts up, it reads the address.text file and gets the
list of available AppServers. It then checks the request for a cookie
(a new one, encoded with the port number and machine some how, maybe
named __PRT__) (see SECURITY ISSUES) identifying the previous
AppServer used for this client. If it exists, it checks to see that
that address is a valid entry from the address.text list (to avoid
stale cookies) and sends the request there. (See more complicated
version of this below, with different mechanism for verifying
AppServer availability) If it doesn't exist, it chooses an available
server randomly. The AppServer is reponsible for adding the cookie
that identifies the machine:port. (So it doesn't have to if there is
no session assigned).
For fail-over, we can have a wimpy little process that is the parent
of the AppServers, and monitors them by sending a special request to
each periodically. If it can't connect to one of them, it kills it
and forks to start another one. Pretty simple little app. It could
also be responsible for the initial starting of the pool of
AppServers. Maybe it's also the only process that actually writes to
address.text, to avoid any concurrent write issues. (that would change
the method I mentioned above)
In order to handle AppSevers limiting the number of requests they can
handle and killing themselves, we'll have to add some more complicated
logic, I think. My first thought on this is that we add a status
check to the initial communication between the Adaptor and the
AppServer. Maybe before sending a request, the Adaptor connects to
the AppServer and asks for the AppServer's status. The AppServer
responds with OK or DYING. If it responds with DYING, the Adaptor
won't send it any new requests, only requests that have gone there
before. (this would work for both cgi and persistent adaptors) (This
could also be the communication mechanism for the monitor process).
When it finally dies, the monitor process can't connect to it anymore
and starts another AppServer on that port. The monitor forks,and if
it's the child, it imports AppServer. We change AppServer slightly to
have a "main" function that takes a machine/port as an optional
parameter. The monitor child calls that function with the specified
port.
The AppServer won't die until it's session cache is empty.
How does that sound?
I think this method will have the lowest impact on the rest of the
code. (Which I consider important) Really the only addition below the
AppServer level is the __PRT__ cookie, and that could be done in the
AppServer (I think). Everything else is at AppServer or above.
SECURITY ISSUES:
I'm worried about putting the port and machine in a cookie. Then any
hacker can get at that port, unless there's a decent firewall set up,
and we can't rely on that, can we? The obvious solution (to me) is to
encode only a count in the cookie, like 2, and have the Adaptor know
to add that number to some configuration parameter to get the actual
port number. Maybe put that configuration number in address.text. How
about that? (We can't use some encoding scheme that relies only on
the cookie value (ie the cookie value is the actual port plus 5
divided by 3 times 7), because this is open source.)
|