Miguel Angel Rasero
2012-06-29
Hi, i am using SymmetricDS 1.7.6 to replicate PostgreSQL databases in Linux,
at the stores the purge jobs works fine and sym_data it's correlated with
sym_outgoing_batch but in the root node i found today that the
min(create_time) of sym_data it's from 2010 and i have 18926276 rows right now
in sym_data and 35551772 in sym_data_event.
sym_outgoing_batch it's fine, have 146245 rows and the min(create_time) it's
from 2012-06-24, i have checked the configuration and are the same at root and
store nodes, what could be happenning and how could i fix it? Thanks!
Regards,
Miguel Angel.
Chris Henson
2012-06-29
Does the log file have errors?
Miguel Angel Rasero
2012-06-29
no, i can't find any error in the logs.
Chris Henson
2012-06-29
Does the purge process run and delete 0 rows? Is the data in sym_data valid
data that had not been sent?
Miguel Angel Rasero
2012-06-29
This is the last process, it seems to delete 0 rows at sym_data, yes.
2012-06-29 19:32:53,737 INFO The outgoing purge process is about to run.
2012-06-29 19:32:53,737 INFO Getting range for outgoing batch
2012-06-29 19:32:53,861 INFO About to purge sym_data_event
2012-06-29 19:32:53,869 INFO Done purging 0 of sym_data_event rows.
2012-06-29 19:32:53,869 INFO About to purge sym_outgoing_batch
2012-06-29 19:32:53,869 INFO Done purging 0 of sym_outgoing_batch rows.
2012-06-29 19:32:53,870 INFO About to purge sym_outgoing_batch_hist
2012-06-29 19:32:54,002 INFO Done purging 0 of sym_outgoing_batch_hist rows.
2012-06-29 19:32:54,002 INFO Getting range for data
2012-06-29 19:32:54,655 INFO About to purge sym_data
2012-06-29 19:35:39,744 INFO Loading properties file from class path resource
2012-06-29 19:35:39,745 INFO Loading properties file from class path resource
2012-06-29 19:35:39,745 WARN Could not load properties from class path
resource : class path resource cannot be opened because it does not exist
2012-06-29 19:35:39,745 INFO Loading properties file from class path resource
2012-06-29 19:35:39,745 WARN Could not load properties from class path
resource : class path resource cannot be opened because it does not exist
2012-06-29 19:35:39,745 INFO Loading properties file from URL
2012-06-29 19:37:24,462 INFO Done purging 0 of sym_data rows.
2012-06-29 19:37:24,469 INFO The outgoing purge process has completed.
2012-06-29 19:37:24,469 INFO The incoming purge process is about to run.
2012-06-29 19:37:24,469 INFO Getting range for incoming batch
2012-06-29 19:37:24,524 INFO About to purge sym_incoming_batch_hist
2012-06-29 19:37:24,708 INFO Done purging 33 of sym_incoming_batch_hist rows.
2012-06-29 19:37:24,708 INFO About to purge sym_incoming_batch
2012-06-29 19:37:24,812 INFO Done purging 33 of sym_incoming_batch rows.
2012-06-29 19:37:24,813 INFO The incoming purge process has completed.
2012-06-29 19:37:24,813 INFO The statistic purge process is about to run.
2012-06-29 19:37:24,839 INFO Purged 4 statistic rows.
2012-06-29 19:37:24,839 INFO The statistic purge process has completed.
Mark Hanes
2012-06-29
Purge looks to be running, but purge will only purge data that has already
been routed and thus is no longer needed. Is the "old" data you expected to be
purged destined for a node that doesn't exist, for example?
Miguel Angel Rasero
2012-06-29
yeah it seems to be the case, the node that it's locking the delete of
sym_data it's a closed store, how should i proceed? marking the node at
sym_node with sync_enabled = 0 and update all sym_data_event batches as
"batched = 1"?
Mark Hanes
2012-06-30
That should do the trick. Be prepared for a long purge run the next time purge
process starts due to the volume of data to be purged.
Miguel Angel Rasero
2012-06-30
Ok, i will try it.
Thanks for help! :)
Miguel Angel Rasero
2012-07-01
This changes don't seems to be enough, still i have 0 rows deleted in sym_data
:(
Chris Henson
2012-07-01
It has been 3 years since I've looked at 1.7 purge code. It looks like you'll
just have to come up with a strategy to delete sym_data_event manually. If you
delete sym_data_event then the purge process should clean up sym_data.
I'd suggest deleting the sym_data_event row in chunks (not all of them at the
same time).
Miguel Angel Rasero
2012-07-01
ok, i will delete sym_data_event rows manually in chunks, thanks.