Name | Modified | Size | Downloads / Week |
---|---|---|---|
Parent folder | |||
tipsy.zip | 2012-04-27 | 5.9 kB | |
Totals: 1 Item | 5.9 kB | 1 |
Manual python module synchronize ================================ Created by: Diethard Jansen, 27-10-2010 This version 0.1.0, 27-4-2012 Added unit tests and repaired some problems that were found during unittesting. Original idea of using file indexes for synchronisation: Jacob van Dijk. "A tiny python module to do massive synchronisation jobs." De python module synchronize can be used for following jobs: 1) Online synchronisation 2) Offline synchronisation 3) Timeline synchronisation ad 1) 'Normal' synchronisation tools use online synchronisation. ad 2) This particular tool can also synchronize remote sites. For this it uses "file index files" which can be send by email. The received result can be used to compare with a locally build file index which will result in an archive that contains the differences and that can be send back to remote site. Ad 3) Of one source directory different snapshots can be taken at different times using "file index files". See following timeline where at different times file indexes have been created at different times. ----1-------2--------3--------4------> time At time 1, 2, 3 and 4 a file index file has been created. Those file index files can be compared to see what has changed to a directory over time. The difference can be delivered when neccesary. At time 1 source dir is copied and delivered to remote site. At any time after 2, 3 or 4 an archive can be created and send to synchronize remote site using created file indexes. An example case taken from a Water company. All fitters carry a laptop with stored on the laptop a directory that contains about 200.000 sketches of houseconnections. Just after they receive the sketches a file index is created. After some time they login again to synchronize their locally stored sketches from the centrally stored library of sketches. Using the timed file index files a relative small archive of changes can be created and used to update the locally stored directory of sketches. Usage of module synchronize in python script: 1 online synchronisation ======================== import synchronize s = synchronize.synchronizer() s.sourceDir = "C:\\data\\pictures" s.targetDir = "E:\\backup\\pictures" s.synchronize() # or using less code: import synchronize s = synchronize.synchronizer() s.synchronize("C:\\data\\pictures", "E:\\backup\\pictures") 2 offline synchronisation ========================= # step 1 create remote index file of remote file that needs to # be synchronized. import synchronize s = synchronize.synchronizer() l_fileIndex = s.fileIndex("D:\\pictures") s.storeObjectToFile(l_fileIndex, "D:\\pictures.idx") # step 2 create at source location an archive to deliver import synchronize s = synchronize.synchronizer() l_fileIndex = s.restoreObjectFromFile("C:\\temp\\pictures.idx") s.sourceDir = "C:\\data\\pictures" s.targetIndex = l_fileIndex s.createArchive("C:\\data\\picture_archive") # Compress the created map holding archive and send to remote location # step 3 synchronize remote directory with received and unpacked archive import synchronize s = synchronize.synchronizer() s.targetDir = "D:\\data\pictures" # add optional argument p_delete to stop unintentially removing # pictures person has added at remote location # in this case the person receiving pictures has decided not # to have any pictures deleted at his side (add only). s.synchonizeUsingArchive("C:\\temp\\picture_archive", p_delete=False): 3 timeline synchronisation ========================== # step 1 create file index at a date. import synchronize s = synchronize.synchronizer() l_fileIndex = s.fileIndex("C:\\data\\pictures") s.storeObjectToFile(l_fileIndex, "C:\\pictures20100801.idx") # step 2, create file index at a later date import synchronize s = synchronize.synchronizer() l_fileIndex = s.fileIndex("C:\\data\\pictures") s.storeObjectToFile(l_fileIndex, "C:\\pictures20100815.idx") # step 3 # Configure synchronizer to use compare on last mutation date # instead of filesize when comparing files with identical # names. import synchronize s = synchronize.synchronizer() # default value = False, using comparison on filesize s.useTime2Compare = True l_firstFileIndex = s.restoreObjectFromFile("C:\\pictures20100801.idx") l_lastFileIndex = s.restoreObjectFromFile("C:\\pictures20100815.idx") s.sourceIndex = l_lastFileIndex s.targetIndex = l_firstFileIndex s.targetDir = ("D:\\data\pictures") s.synchronize() or for offline purposes.. replace last two statements with: s.createArchive("C:\\data\\picture_archive20100815") and send compressed result to person who can use archive to synchronize target directory.