Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

resource forks

2006-05-25
2013-04-30
  • The interface looks nice. What about rsync's issues with resource forks/extended attributes (no modification times). Are you doing anything to make sure these files get synched correctly?

     
    • Miles Wu
      Miles Wu
      2006-05-25

      We are (sort of) aware of Apple's messed up implementation of HFS resource forks in the included version of rsyc So far we have only discovered the problem when you asked it to sync with extended attributes and to always checksum. This caused a buffer overflow error in rsync itself as it is attempting to checksum non-existant 'files'. So at the moment we have prevented these two options being set at the same time. So in short, we haven't really done anything about it at all. Correct me if I am wrong, but I believe that rsync, by default, conservatively copies the resource forks again, even if they haven't changed, as they have no modification time of their own.

      So far we have tried to use the following modified versions of rsync, all of which claim to fix these problems. However none of them appear to work successfully on my machine. Some segfault and some just ignore the extended attributes. This is possibly due to my machine being an Intel Mac and the modifications may be endian specific.

      http://www.lartmaker.nl/rsync/
      http://www.onthenet.com.au/~q/rsync/
      http://hfsrsync.darwinports.com/

      If you have any suggestions or advice for us, please don't hesitate to reply. Oh, and the UI designer thanks you.

       
    • I don't have any useful suggestions. My understanding is that changes to the extended attributes do not result in a change to the file's modification time, even though the changes may be 'significant' to the file.

      My plan was to try running apple's rsync with 'checksum everything' turned on, and see if it was still fast enough to use for overnight backups. If it just crashes like you say it does, then that's probably not going to work out to well.

      All the stuff I've read about this says that apple rsync will always copy the forked files. So the only problem is that your incrementals will bloat with spurious extra versions. How bad this is depends on how many files you have with resource forks (and how big they are).

      Do you have a sense of how many there are on a typical mac?

       
    • Basically, we won't be able to get checksums and resource forks to play nicely together (at least in the near future).

      For some idea of how much space resource forks take up, I'd suggest Amit Singh's article of HFS+ fragmentation: http://www.kernelthread.com/mac/apme/fragmentation/ The sort of figures he was seeing on his systems were ranging from 1.25% to 0.09% of all the data being resource forks. Typically for most user data at least, I wouldn't have though resource forks would be a huge amount of data.

      Hope this helps.

       

  • Anonymous
    2011-05-17

    Any reason why you wouldn't compile rsync 3.0.7 or 3.0.8 with patches so the extended attributes are included?

     

  • Anonymous
    2011-05-17

    BTW, the included version of rsync on OSX 10.5.8 is 2.6.9 v29.  As rsync with hfs+ support includes xattr, the version 2.6.9 is a security risk.  So perhaps you should be including your own rsync compiled into arRsync?