I was continuously searching through forums to learn and setup data synchronization.
What I'll be doing is pulling from our software Solarwinds (you may be able to do this using AD too) and creating a report / csv that is generated regularly.
This is a test to setup a person (subtree of a contact).
iTop side:
admin tools > synchronization data sources > new (name it whatever).
properties: Status>implementation, target class > person contact to notify alerts this person when an error in the sync happens.
searches and reconciliation: reconciliation policy > use the attributes, action on zero>create, action on one>update, action on many>error.
im still fiddling with deletion rules, so for now just set users allowed > nobody and delete policy > ignore.
hit apply and it should fill in a "data table" for you. You can also look in the URL and find where &id=#, where the # is a number specific to that table and you will use it in a future step.
Under attributes tab:
reconciliation: Last name and organization checked off
update: checked off the fields i put in my csv (last name, org, email, first name)
update policy: i unlocked everything for testing purposes.
Reconciliation Key: next to org id i changed to name
Now on the itop server side:
Here is my csv file. I am contemplating having the primary_key field be able to sync to our solarwinds account's primary_key field. This would be optimal because if lets say their last name changes iTop will create a new record and then dete the existing record because the primary_key didnt match. This idea can be extended to servers / equipment as well.
Just make sure the folder drills down properly to your synchro_import, synchro_exec and your csv file.
data_sources=#, the # is the same number you found earlier in the url that is specific to the sync table created earler.
Run this command and it should pull everything and sync it.
Some things: Im a bit hesitant on the deletion rules, I think youre only able to "unlock" something to be deleted, but it wont actually delete (from looking at a definition via itop). I also highly suggest finding a good primary_key. If you change something that is the primary_key of a field it will create a new record and delete the old one instead of properly updating the name / etc.
I also logged into the mysql server on our itop server to verify how things looked on the table.
go to command line:
Mysql -u root // or whatever credentials?
Use itopv23; //or whatever your databases name is
Show tables; // shows every table. keep in mind theres a virtualmachines table AND a view_virtualmachines
Select * from person; //displays the persons table.
DELETE FROM person
WHERE id = '1'; //just an idea of how you can delete a specific row.
select * from view_virtualmachine where name = 'thisname'; //specifies in that table what to view (can search by farm etc).
this will help you troubleshoot some issues you have with tables to see the records.
if you have viewing issues select the cmd prompt top bar and go to properties > layout > screen buffer size 600 / 900 and window size > 180 / 70 is what i have so i can see the tables clearly.
hoping i didnt leave anything out. any comments regarding the deletion process would be appreciated!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So I see two different definitions from the links above:
1) This section defines how the deletion of the synchronized objects are handled:
by the end-users in iTop,
by the synchronization system itself.
Users Allowed - Which end-users are allowed to delete synchronized objects in iTop.
Nobody when an object is synchronized, nobody is allowed to delete it in
iTop. only the synchronization system can delete it.
Administrators only - only administrators are allowed to delete a synchronized object.
Everybody allowed to delete such objects the normal access rules apply to
such synchronized objects.
Full load interval - The duration between two executions of the synchronization which are
considered as having loaded (touched) all replicas. When a replica has not
been touched for a duration greater than this value, it is considered as
obsolete.
Delete Policy - What to do when a replica becomes obsolete. Ignore do nothing, the
associated object remains as is in iTop.
Delete - Delete the associated object in iTop (and the replica in the data table).
Update - Update the associated object as specified by the Update rules (see below).
Update - the Delete Update the associated object as specified by the Update rules
(see below), then after the Retention Duration, delete the object.
Update rules - A list of values separated by semi-colons, of the form attribute_code:value to
specify which attribute of the associated object to set and to which value.
Example: status:obsolete;description:no longer synchronized.
Retention duration - The duration between the update of the associated object and its deletion
when the Delete Policy is set to Update then delete.
2) Deleting records from the sync data table will not delete the objects. It will just detach the objects from the synchronization mechanism. Users will have the right to edit or delete the objects exactly as if they were created from within the UI. Re-attaching the objects to the synchronization requires both to setup the data source so that the reconciliation can occur, then re-introduce the records into the sync data table. See how the deletion works hereabove. The synchronization will delete the records from the synch data table for you.
Above, those are copy and paste answers from the links provided.
Number 1 seems like it actually would delete the record. But from the definition provided, delete just means it "unlocks"? So this CI would still exist in the database correct? Is there any way to view when the synchronization "deletes" (aka unlocks) a CI? Because it looks like I will have to manually go in and delete the CI.
Last edit: Darcy Dressell 2016-09-26
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I had this confusion also. But when you look at the SynchroReplicas, you will see the field "Object Created", and that's an important one. The synchro will only delete objects that it has created itself. The others will be ignored and unlocked.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Oh okay that makes sense. I'll have to do some testing because what we'll be doing is synchronizing our servers into iTop, but I think what we'll have to do is manually push in a CSV file for our services on that server (since you can't have multiple primary keys).
Thanks so much!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you. All understand in synchronization in ITOP. I generally have confusion with the synchronization of data. I work in the company and constantly there are any failures. I need to synchronize information. Can switch to other software. The only thing that stops the change of software - in the company need data from Oulook. And in any way we can not collect all this information and synchronize. If someone tells me any service I will be grateful.
Last edit: Dmitry 2017-06-12
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi All,
I was continuously searching through forums to learn and setup data synchronization.
What I'll be doing is pulling from our software Solarwinds (you may be able to do this using AD too) and creating a report / csv that is generated regularly.
This is a test to setup a person (subtree of a contact).
iTop side:
admin tools > synchronization data sources > new (name it whatever).
properties: Status>implementation, target class > person contact to notify alerts this person when an error in the sync happens.
searches and reconciliation: reconciliation policy > use the attributes, action on zero>create, action on one>update, action on many>error.
im still fiddling with deletion rules, so for now just set users allowed > nobody and delete policy > ignore.
hit apply and it should fill in a "data table" for you. You can also look in the URL and find where &id=#, where the # is a number specific to that table and you will use it in a future step.
Under attributes tab:
reconciliation: Last name and organization checked off
update: checked off the fields i put in my csv (last name, org, email, first name)
update policy: i unlocked everything for testing purposes.
Reconciliation Key: next to org id i changed to name
Now on the itop server side:
Here is my csv file. I am contemplating having the primary_key field be able to sync to our solarwinds account's primary_key field. This would be optimal because if lets say their last name changes iTop will create a new record and then dete the existing record because the primary_key didnt match. This idea can be extended to servers / equipment as well.
person.csv
primary_key;org_id;name;first_name;email
LastName;CompanyName;LastName;FirstName;emailAddress@test.com
Then i created a command document that contained:
php E:\xampp\htdocs\itop\synchro\synchro_import.php --auth_user=admin --auth_pwd=pwhere --csvfile=E:\itopexec\persons.csv --data_source_id=2 --output=summary
php E:\xampp\htdocs\itop\synchro\synchro_exec.php --auth_user=admin --auth_pwd=pwhere --data_sources=2
pause
Just make sure the folder drills down properly to your synchro_import, synchro_exec and your csv file.
data_sources=#, the # is the same number you found earlier in the url that is specific to the sync table created earler.
Run this command and it should pull everything and sync it.
Some things: Im a bit hesitant on the deletion rules, I think youre only able to "unlock" something to be deleted, but it wont actually delete (from looking at a definition via itop). I also highly suggest finding a good primary_key. If you change something that is the primary_key of a field it will create a new record and delete the old one instead of properly updating the name / etc.
I also logged into the mysql server on our itop server to verify how things looked on the table.
go to command line:
Mysql -u root // or whatever credentials?
Use itopv23; //or whatever your databases name is
Show tables; // shows every table. keep in mind theres a virtualmachines table AND a view_virtualmachines
Select * from person; //displays the persons table.
DELETE FROM person
WHERE id = '1'; //just an idea of how you can delete a specific row.
select * from view_virtualmachine where name = 'thisname'; //specifies in that table what to view (can search by farm etc).
this will help you troubleshoot some issues you have with tables to see the records.
if you have viewing issues select the cmd prompt top bar and go to properties > layout > screen buffer size 600 / 900 and window size > 180 / 70 is what i have so i can see the tables clearly.
hoping i didnt leave anything out. any comments regarding the deletion process would be appreciated!
Hi Darcy,
Thanks for sharing your experience. Did you havea look at the wiki? There are 3 articles related to data synchronization, including a simple "how to" about synchronizing persons:
Overview: https://wiki.openitop.org/doku.php?id=2_3_0:advancedtopics:data_synchro_overview
Simple "How to": https://wiki.openitop.org/doku.php?id=2_3_0:advancedtopics:synchro_howto_personsfromldap
Reference: https://wiki.openitop.org/doku.php?id=2_3_0:advancedtopics:data_synchronization
So I see two different definitions from the links above:
1) This section defines how the deletion of the synchronized objects are handled:
Users Allowed - Which end-users are allowed to delete synchronized objects in iTop.
Nobody when an object is synchronized, nobody is allowed to delete it in
iTop. only the synchronization system can delete it.
Administrators only - only administrators are allowed to delete a synchronized object.
Everybody allowed to delete such objects the normal access rules apply to
such synchronized objects.
Full load interval - The duration between two executions of the synchronization which are
considered as having loaded (touched) all replicas. When a replica has not
been touched for a duration greater than this value, it is considered as
obsolete.
Delete Policy - What to do when a replica becomes obsolete. Ignore do nothing, the
associated object remains as is in iTop.
Delete - Delete the associated object in iTop (and the replica in the data table).
Update - Update the associated object as specified by the Update rules (see below).
Update - the Delete Update the associated object as specified by the Update rules
(see below), then after the Retention Duration, delete the object.
Update rules - A list of values separated by semi-colons, of the form attribute_code:value to
specify which attribute of the associated object to set and to which value.
Example: status:obsolete;description:no longer synchronized.
Retention duration - The duration between the update of the associated object and its deletion
when the Delete Policy is set to Update then delete.
2) Deleting records from the sync data table will not delete the objects. It will just detach the objects from the synchronization mechanism. Users will have the right to edit or delete the objects exactly as if they were created from within the UI. Re-attaching the objects to the synchronization requires both to setup the data source so that the reconciliation can occur, then re-introduce the records into the sync data table. See how the deletion works hereabove. The synchronization will delete the records from the synch data table for you.
Above, those are copy and paste answers from the links provided.
Number 1 seems like it actually would delete the record. But from the definition provided, delete just means it "unlocks"? So this CI would still exist in the database correct? Is there any way to view when the synchronization "deletes" (aka unlocks) a CI? Because it looks like I will have to manually go in and delete the CI.
Last edit: Darcy Dressell 2016-09-26
I had this confusion also. But when you look at the SynchroReplicas, you will see the field "Object Created", and that's an important one. The synchro will only delete objects that it has created itself. The others will be ignored and unlocked.
Oh okay that makes sense. I'll have to do some testing because what we'll be doing is synchronizing our servers into iTop, but I think what we'll have to do is manually push in a CSV file for our services on that server (since you can't have multiple primary keys).
Thanks so much!
Where do you see this field "SynchroReplicas" -> "Object Created" ?
Thank you. All understand in synchronization in ITOP. I generally have confusion with the synchronization of data. I work in the company and constantly there are any failures. I need to synchronize information. Can switch to other software. The only thing that stops the change of software - in the company need data from Oulook. And in any way we can not collect all this information and synchronize. If someone tells me any service I will be grateful.
Last edit: Dmitry 2017-06-12