synchro_exec.php: Replicas number exploding

Klaus Muth
2012-05-07
2013-08-11
  • Klaus Muth

    Klaus Muth - 2012-05-07

    Hi.
    Tried to do User sync with ldap but my number of Replicas is exploding:

    Three consecutive cycles of filling synchro_data_userldap_1 and doing synchro_exec.php yield:

    Replicas: 561
    Replicas touched since last synchro: 561
    Objects deleted: 0
    Objects deletion errors: 0
    Objects obsoleted: 0
    Objects obsolescence errors: 0
    Objects created: 0 (0 warnings)
    Objects creation errors: 0
    Objects updated: 0 (0 warnings)
    Objects update errors: 0
    Objects reconciled (updated): 0 (0 warnings)
    Objects reconciled (unchanged): 0 (0 warnings)
    Objects reconciliation errors: 0
    Replica disappeared, no action taken: 0

    Replicas: 748
    Replicas touched since last synchro: 748
    Objects deleted: 0
    Objects deletion errors: 0
    Objects obsoleted: 0
    Objects obsolescence errors: 0
    Objects created: 0 (0 warnings)
    Objects creation errors: 0
    Objects updated: 0 (0 warnings)
    Objects update errors: 0
    Objects reconciled (updated): 0 (0 warnings)
    Objects reconciled (unchanged): 0 (0 warnings)
    Objects reconciliation errors: 0
    Replica disappeared, no action taken: 0

    Replicas: 935
    Replicas touched since last synchro: 935
    Objects deleted: 0
    Objects deletion errors: 0
    Objects obsoleted: 0
    Objects obsolescence errors: 0
    Objects created: 0 (0 warnings)
    Objects creation errors: 0
    Objects updated: 0 (0 warnings)
    Objects update errors: 0
    Objects reconciled (updated): 0 (0 warnings)
    Objects reconciled (unchanged): 0 (0 warnings)
    Objects reconciliation errors: 0
    Replica disappeared, no action taken: 0

    With the difference between cycles (187) being the number of entries in the synchro_data_userldap_1 table

    My script to fill the synchro_data_userldap_1 table just updates - no inserts were done.

    Could please someone shed some light and point me in the right direction? Is everything ok? So what happens after some 10^x cycles when the Replicas number reaches maxint? Am I doing something wrong?

    Klaus

     
  • Klaus Muth

    Klaus Muth - 2012-05-09

    Wow.
    Patch won't work by patch < synchrodatasource.class.inc.php.patch, so I applied it by hand, changing

            $sTriggerInsert = "CREATE TRIGGER `{$sTable}_bi` BEFORE INSERT ON `$sTable`";
            $sTriggerInsert .= "   FOR EACH ROW";
            $sTriggerInsert .= "   BEGIN";
            $sTriggerInsert .= "      INSERT INTO `{$sReplicaTable}` (`sync_source_id`, `status_last_seen`, `status`) VALUES ({$this->GetKey()}, NOW(), 'new');";
            $sTriggerInsert .= "      SET NEW.id = LAST_INSERT_ID();";
            $sTriggerInsert .= "   END;";
            $aResult['bi'] = $sTriggerInsert;
    

    to

            $sTriggerInsert = "CREATE TRIGGER `{$sTable}_bi` BEFORE INSERT ON `$sTable`";
            $sTriggerInsert .= "   FOR EACH ROW";
            $sTriggerInsert .= "   BEGIN";
            $sTriggerInsert .= "      SET NEW.id = (SELECT IFNULL(MAX(id)+1, 1) FROM `{$sReplicaTable}`);";
            $sTriggerInsert .= "   END;";
            $aResult['bi'] = $sTriggerInsert;
            $sTriggerInsert = "CREATE TRIGGER `{$sTable}_ai` AFTER INSERT ON `$sTable`";
            $sTriggerInsert .= "   FOR EACH ROW";
            $sTriggerInsert .= "   BEGIN";
            $sTriggerInsert .= "      INSERT INTO `{$sReplicaTable}` (`id`, `sync_source_id`, `status_last_seen`, `status`) VALUES (NEW.id, {$this->GetKey()}, NOW(), 'new');";
            $sTriggerInsert .= "   END;";
            $aResult['ai'] = $sTriggerInsert;
    

    Tried it, but still the #of replicas is increasing.

    How do I get rid of the excess replica entries? Just delete all priv_sync_replica entries? Or delete those entries with status='new'? Or better forget about it and live with those entries?

     
  • DemonCcC

    DemonCcC - 2012-05-09

    This patch works when you create/recreate new synchro data resources.
    I forgot to mention that you need to recreate your synchro data resources or modify the triggers of the synchro_data_XXX tables.

     
  • jonathon_ucb

    jonathon_ucb - 2013-08-09

    Hello,

    I'm having the same problem with the number of replicas increasing with every synchro job using iTop 2.0.1. I manually applied the above patch, deleted and re-created all of my data sources but still the replicas are doubling with each job. This is particularly troublesome since I have one data source with 80,000 records.

    Any advice?

     
  • jonathon_ucb

    jonathon_ucb - 2013-08-11

    I think I figured out the issue for me by reading some more of the documentation. The replicas seem to be working now. I had to:

    1. Delete all of my data source synch tables
    2. Implement the bug fix for synchrodatasource.class.inc.php
    3. Create the data source synch tables
    4. Set the reconciliation policy to primary_key
    5. Manually set the index for primary_key on each table to be UNIQUE

    It seems that step 5 should be handled by the iTop UI when setting the reconciliation policy. Currently it sets the primary_key index to be NON-UNIQUE.

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks