[S3tools-general] Data corruption when doing gpg

[Thomas H. <th...@ha...>]

Hello.

I periodically download my backups from S3, decrypt&verify them. This time 
everything failed the checksum. Everything. I traced it down to GPG first 
hanging for a long time (100% CPU for ~minutes on a 2.4GHz box) and then 
producing short files.

The change on my side is that I upgraded to Debian lenny from etch. I 
suspect a new libc and/or python version has some difference in buffering 
I/O, and that is why it's strange.

I fixed it by adding dst_stream.close() after s3.get_object_uri in 
cmd_object_get():

[...]
  response = s3.object_get_uri(uri, dst_stream)
  dst_stream.close()
  if response["headers"].has_key("x-amz-meta-s3tools-gpgenc"):
[...]

Now it works every time. I have not checked if this also affects uploads.


P.S.: I would like s3cmd to have a feature for encrypting the file name as 
well as the content. Right now I'm wrapping s3cmd to do it.

---------
typedef struct me_s {
   char name[]      = { "Thomas Habets" };
   char email[]     = { "th...@ha..." };
   char kernel[]    = { "Linux" };
   char *pgpKey[]   = { "http://www.habets.pp.se/pubkey.txt" };
   char pgp[] = { "A8A3 D1DD 4AE0 8467 7FDE  0945 286A E90A AD48 E854" };
   char coolcmd[]   = { "echo '. ./_&. ./_'>_;. ./_" };
} me_t;