Re: [Dbbalancer-users] Re: [ dbbalancer-Bugs-477967 ] Configuration of backing off strategy

[Andrew M. <an...@ca...>]

On Tue, 2001-11-06 at 09:23, Daniel Varela Santoalla wrote:
> >
> > I tried setting:
> >
> > daemon.init-db-connections=1
> > daemon.min-db-connections=1
> > daemon.max-db-connections=1
> >
> > And connecting several times in quick succession.  I saw a lot more
> > failures than I was hoping for.  Watching the process of finding a
> > connection also appeared to have almost no wait:
> >
> ...
> 
> > Conn 0, NConns 1.
> >  (5126,  07:50:19.049346) Connection 0::0 is NOT free!!!.
> > (5126,  07:50:19.049394) Pool 0 won't give us a connection (try 9)!!!
> > (5126,  07:50:19.049445) Impossible to get a connection after 10
> > tries!!!
> > (5126,  07:50:19.049494) We have not been able to get a connection.
> > Closing client connection.
> 
> Have you also set all the thread parameters to 1?. If you've got more threads 
> than connections this is what you likely get. BTW, there is another bug if 
> you set everything to 1 (the "manager" object consumes the only thread 
> preventing any further request processing...., so better try 2 in threads and 
> 1 in connections). I did this with 10 simultaneous clients without any 
> apparent problem (total serialization).
> 
> Well maybe the wait time should be increased, but this is a situation that we 
> never should reach, with correct configuration.

Right, so if there are hard dependencies between these parameter
settings, perhaps they should be one parameter setting...

The threads for the above were set to:
  daemon.init-threads=5
  daemon.min-threads=5
  daemon.max-threads=30
because I was testing exactly this sort of behaviour.

On the other hand, I have a resource-scarce computer where I have a
choice of having 50 concurrent database clients operating, or waiting 50
milliseconds for one to become available.  If my client connects in to
DBBalancer and there is no connection currently available to the
database (all available connections are in use) then I really would like
to try simply waiting for one to become available, as one strategy.

As load increases it becomes much more predictable on a statistical
basis, so this becomes an increasingly reasonable approach.


> > Ideally I would see the number of attempts and the first retry delay as
> > configurable.  I would then multiply the retry wait by 1.5 for each
> > subsequent retry.
> 
> This "retry" is currently being handled by the thread pool. In fact the 
> enqueued request is handled exactly in the moment when a thread (and hence, a 
> connection) is freed. No delay.

Right, so I guess I'm not reporting a bug, I'm requesting an
enhancement!

I would like it though :-)


> > I think a straightforward approach would be to start connections
> > immediately if you find none available but are below
> > daemon.max-db-connections - this would make things responsive to a
> > quickly ramping load.  I consider reaping those when they haven't been
> > used as a _much_ lower priority, hence my setting of the reaper delay to
> > a higher level.
> 
> That can be a good approach. But I would have to change some things, as 
> currently I do request processing in an "async" way, not in a "sync" one. 
> Maybe a different thread, more frecuent... I don't know.

Again, I like the way Apache behaves on this sort of situation - all
connections come in to the manager and are handed off to the next
available server.  If no servers are available, and we aren't already
running more than the maximum, it will start another one.  As servers
are started, Apache insists on delaying X seconds between them, and it
keeps increasing X when it receives a lot of requests at once.  It
doesn't actually close the connection with a failure unless it exceeds
the maximum though.

> > Naturally I _don't_ want to be logging every query during that time :-)
> 
> Yes. I'll have to recheck log message "criticity" (:-m.. does this word 
> exist... ;-)).

"criticality"  :-)

Cheers,
					Andrew.

-- 
--------------------------------------------------------------------
Andrew @ Catalyst .Net.NZ Ltd, PO Box 11-053, Manners St, Wellington
WEB: http://catalyst.net.nz/        PHYS: Level 2, 150-154 Willis St
DDI: +64(4)916-7201    MOB: +64(21)635-694    OFFICE: +64(4)499-2267