[Sqlrelay-discussion] infinite loop

[Devananda <kar...@ya...>]

Dear List,

sqlrelay is regularly getting stuck in an infinite reconnect loop on 
both servers on which I am running it right now. This seems to happen 
/most often/ after hours of inactivity, but not always. I've spent a 
week tracking this down, and have found where the MySQL C API is 
returning an unexpected result, and sqlrelay goes into an infinite loop 
because of it.

First, the platform; sqlrelay pulled from CVS a few weeks ago, Fedora 
4/5, and MySQL 5.0.18-5.0.22.

The loop itself happens inside the sqlrconnection_svr::handleQuery 
function, where the inline comments clearly state "loop here to handle 
down databases" :) The problem is that the MySQL C API (erroneously?) 
returns an error message (CR_SERVER_GONE_ERROR) at mysqlconnection.C:542

if ((queryresult=mysql_stmt_execute(stmt))) {

As a result, sqlrconnection_svr::processQuery returns FALSE, which 
triggers sqlrconnection_svr::handleError, which can not returnError() 
because the database appears to be down, and therefor calls reLogIn(). 
However, the database is not actually down, and sqlrelay reconnects to 
it easily, tries to rerun the initial query, and gets the same return 
value of CR_SERVER_GONE_ERROR, thus beginning the loop again.

Running sqlr-status during this time shows number of connections 
skyrocketing, and asking MySQL to "SHOW STATUS" shows the number of 
client connections (Aborted_clients and Connections) to be increasing as 
well. "SHOW PROCESSLIST" also displays sqlrelay's connections, which are 
being established and closed as fast as the server is able.

This looks as though it could be caused by the bug reported at 
http://bugs.mysql.com/bug.php?id=19927.

I have added some code to detect and break out of the infinite loop, 
hoping that would patch sqlrelay while MySQL fixes their bugs, however, 
simply breaking out of the loop does not sufficiently fix what ever is 
wrong. Any further debugging is beyond my abilities :(

If anyone can help, it would be very appreciated. We are on a pretty 
tight development schedule, and this is really slowing us down. If 
anyone needs to see a sqlrelay debug file, let me know and I'll send it 
off-list.



Best Regards,
Devananda van der Veen