Sync Errors: s3cmd -Ver 0.9.5, Python2.5

dsbcpas
2008-02-25
2013-02-19
  • dsbcpas
    dsbcpas
    2008-02-25

    Shows files to upload or delete, but does not upload them. S3 directory located at /usr/bin/S3. I am guessing I miss located something else?? Thank you.

    Traceback (most recent call last):
      File "/usr/bin/s3cmd", line 741, in <module>
        cmd_func(args)
      File "/usr/bin/s3cmd", line 402, in cmd_sync
        attr_header = _build_attr_header(src)
      File "/usr/bin/s3cmd", line 287, in _build_attr_header
        val = pwd.getpwuid(st.st_uid).pw_name
    KeyError: 'getpwuid(): uid not found: 1001'
    Traceback (most recent call last):
      File "/usr/bin/s3cmd", line 741, in <module>
        cmd_func(args)
      File "/usr/bin/s3cmd", line 402, in cmd_sync
        attr_header = _build_attr_header(src)
      File "/usr/bin/s3cmd", line 287, in _build_attr_header
        val = pwd.getpwuid(st.st_uid).pw_name
    KeyError: 'getpwuid(): uid not found: 8822'
    Traceback (most recent call last):
      File "/usr/bin/s3cmd", line 741, in <module>
        cmd_func(args)
      File "/usr/bin/s3cmd", line 402, in cmd_sync
        attr_header = _build_attr_header(src)
      File "/usr/bin/s3cmd", line 287, in _build_attr_header
        val = pwd.getpwuid(st.st_uid).pw_name
    KeyError: 'getpwuid(): uid not found: 112'

     
    • Michal Ludvig
      Michal Ludvig
      2008-02-25

      It looks like you've got files owned by UIDs with no corresponding user. That should indeed work. I'll fix it asap.

       
    • dsbcpas
      dsbcpas
      2008-02-28

      Looks like Ver 0.9.6 fixed the missing UIDs problem.

      Follow up Questions:
      - Any way in the configuration file to provide default user and group names or numbers when they are missing?

      - I also received a few internal errors, below, with this update, no other debug information. Appears to be reproducible with some debug information. It continues to run with the error. I will attempt to locate better information for you.

      ERROR: S3 error: 500 (Internal Server Error): InternalError

      -Is there any more documentation on sync for example is it file time stamp driven and any way to specify files or subdirectories to exclude?

      Thank you,
      Scott

       
      • Michal Ludvig
        Michal Ludvig
        2008-02-28

        > Any way in the configuration file to provide default user
        > and group names or numbers when they are missing?

        No. It stores the numeric UID/GID instead.

        > ERROR: S3 error: 500 (Internal Server Error): InternalError

        Interesting, I have never seen one. What s3cmd command did you run?

        > Is there any more documentation on sync for example is it
        > file time stamp driven

        For now it's MD5-sum driven as that's the only thing I get in bucket listing. For timestamp or other attribute-driven sync we'd have to perform lots of queries, basically one for each object in a bucket to get the headers. It's on my TODO list to solve it somehow, either have an option to force this headers retrieval or have some cache or aggregated index file or something similar. That will actually help with GPG encrypted sync as well, hmm, I'll think about it.

        > any way to specify files or subdirectories to exclude?

        No, not yet. Shouldn't be hard to do though. I'll put it on my TODO list.

        Michal

         
    • dsbcpas
      dsbcpas
      2008-02-28

      Michal, Thank you for this tool. We have a four tiered back up protocol and hope to dump the old unreliable "tape" (tier 2) for S3. In the last (I am an old timer) crash and burn in December (which included smoke), we lost some data that we still have not been able to retrieve it from tape.

      I started a new thread on: ERROR: S3 error: 500 (Internal Server Error): InternalError

      With regard to GPG encrypted sync, that was next on my list to attempt next.
      Am I correct to assume from your comment that it would not work because the md5sum comparison is looking at the local file before encryption and comparing it with uploaded after encryption file version?

      Scott