we have a problem with Symmetric on MySQL, this is what happened:
We had e server node with some batches (no more than 15-20 and mostly heartbeats) pending for a node that was offline. Once the node has been connected all batches were sent but MySQL went into 'Too many connections' error (more than 151 connections). After restarting symmetric service all came back to normality.
The strange thing is that symmetric was configured as follows (conf/symmetric.properties) and for what i understand it shouldn't open more than 15 connections. Am i wrong? The application running on client nodes (a selling application) takes no more than 10-15 connections.
Forgot to mention: All symmetric engines are running on same server. There are 2 client nodes and a server one.
You are correct in that SymmetricDS should not use more than 15 connections. Was there anything suspicious going on in your log file?
Hi, this is the log in the moment i connected the node.
Seems normal to me, except for the "too many connections" error.
We had the same problem few days ago with almost the same situation.
The slave nodes where unreachable for a network fault (the slaves have been offline for about 8 hours), once we resolved the problem on slave PCs (symmetric was still running on master PC) the databases have become unreachable (too many connections). Rebooting symmetric resolved the problem.
Our symmetric.properties file have the same configuration as Marco, in normal conditions Symmetric does its work with no problems and it uses 9-10 connections.
Why does Symmetric use all available MySQL connections when recovering from a network fault? And how can we resolve this problem?
Thanks for the detailed description. I think I recognize this issue.
I think this is related to this:
I have a patch from several years ago, that I will apply to SymmetricDS that will go in 3.6.6.
Thanks for the quick response.
Do you have any idea how we can get around the problem?
My patch is available in this snapshot.
The patch was to ResettableBasicDataSource in the symmetric-jdbc jar file. You could probably extract the class file (it hasn't changed for some time) and put it under your patches directory in a org/jumpmind/db/util folder.
I'm curious to see if it resolves your issue.
I'm curious to see if i resolve ...
I'm sorry but i don't understand what to do.. do I have to put the class file in the patches (org/jumpmind/db/util) folder of Symmetric? Or should I update the whole Jar file (Symmetric-jdbc-3.6.6-SNAPSHOT.jar)?
extract ResettableBasicDataSource.class from the jar file and put in the patches directory so you end up with:
btw - what version of symmetricds are you running?
We are running Symmetric 3.6.1
In that case, it would be pretty safe to just go ahead and do a complete upgrade to the snapshot (versus patching). Up to you ...
I try and then let you know
I'm not very confident about use this snapshot in a production environment, is it stable enough? Sorry if i ask, do you have any ETA for stable 3.6.6?
The release of 3.6.6 will be early next week. The current snapshot (as of last night) should be stable enough to test with. All tests are passing.
Thank you vary much... I think we will wait for final release.