Currently I'm having a problem with SymmetricDS in corp-> store where triggers are being activated and data is being added to sym_data_event but SymmetricDS isn't creating the batches in sym_outgoing_batches.
In case of a reload from corp -> store the batches are created correctly, but the batches in common mode are not being created.
I'm not sure what caused this but the only recent activity we have done was cleaning up the sym_outgoing_batches, sym_data and sym_data_event table which were on 100GB+ between the three of them.
Version used in the corp node is 3.8.28
Any help would be appreciated.
Thanks!
Last edit: Alvaro Reyes 2017-12-01
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Try to run purge proces from command line from bin directory of symmetric or manually delete all records from sym_data and sym_data_event where sym_ougoing_batch has status set to OK. Also you can try to delete all record in sym_data_gaps table. Restart symm service after that.
And is your purge proces active? +100GB for this 3 tables this is not normal.. these table records should be deleted periodically once in a day or more often, depends on your settings and how many records you process every day.
I dont know where problem is, but two or three times I had some similar problem that no new outgoing batch and even data_event were not created and had to delete all records in sym_data, data_event, outgoting_batch and also sym_data_gap and restart services. Then it start working again.
But I would say that your tables sym_outgoing_batches, sym_data and sym_data_event are now probably too huge and symmetric can´t work properly or work very slow so no new outoging_batch are crated. Also try to check your log for some errors or some warning.
Also I strongly recommend to run sweep proces in your database if you really delete +100GB of data and after that do backup/restore, again I had similar experience (dont know which type od database you using) that after deleting many GB of data, symmetric start to work very slow or even stop working.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
A note on the creation of outgoing batches... Outgoing batches are created by the "Routing" job. The Routing job looks at the sym_data_gap table to determine which data to select from sym_data. So given:
sym_data_gap start_id=1, end_id=5000001
and
sym_data that has data_id's, of 1,2,3:
Routing will run and create sym_data_events for 1,2,3, with corresponding rows in sym_outgoing_batch.
Sometimes the sym_data_gap can get out of range with the data_id's in sym_data. In that case you can safely stop SymmetricDS, truncate sym_data_gap, and restart.
Hope this helps troubleshoot.
Mark
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
Currently I'm having a problem with SymmetricDS in corp-> store where triggers are being activated and data is being added to sym_data_event but SymmetricDS isn't creating the batches in sym_outgoing_batches.
In case of a reload from corp -> store the batches are created correctly, but the batches in common mode are not being created.
I'm not sure what caused this but the only recent activity we have done was cleaning up the sym_outgoing_batches, sym_data and sym_data_event table which were on 100GB+ between the three of them.
Version used in the corp node is 3.8.28
Any help would be appreciated.
Thanks!
Last edit: Alvaro Reyes 2017-12-01
Try to run purge proces from command line from bin directory of symmetric or manually delete all records from sym_data and sym_data_event where sym_ougoing_batch has status set to OK. Also you can try to delete all record in sym_data_gaps table. Restart symm service after that.
And is your purge proces active? +100GB for this 3 tables this is not normal.. these table records should be deleted periodically once in a day or more often, depends on your settings and how many records you process every day.
I dont know where problem is, but two or three times I had some similar problem that no new outgoing batch and even data_event were not created and had to delete all records in sym_data, data_event, outgoting_batch and also sym_data_gap and restart services. Then it start working again.
But I would say that your tables sym_outgoing_batches, sym_data and sym_data_event are now probably too huge and symmetric can´t work properly or work very slow so no new outoging_batch are crated. Also try to check your log for some errors or some warning.
Also I strongly recommend to run sweep proces in your database if you really delete +100GB of data and after that do backup/restore, again I had similar experience (dont know which type od database you using) that after deleting many GB of data, symmetric start to work very slow or even stop working.
A note on the creation of outgoing batches... Outgoing batches are created by the "Routing" job. The Routing job looks at the sym_data_gap table to determine which data to select from sym_data. So given:
sym_data_gap start_id=1, end_id=5000001
and
sym_data that has data_id's, of 1,2,3:
Routing will run and create sym_data_events for 1,2,3, with corresponding rows in sym_outgoing_batch.
Sometimes the sym_data_gap can get out of range with the data_id's in sym_data. In that case you can safely stop SymmetricDS, truncate sym_data_gap, and restart.
Hope this helps troubleshoot.
Mark