Menu
▾
▴
s3tools-bugs
|
From: Francois B. <fra...@wa...> - 2009-04-17 15:24:35
|
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the following lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Problem: ExpatError: mismatched tag: line 5, column 2
S3cmd: 0.9.9
Python: 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) [GCC 4.2.3 (Ubuntu
4.2.3-2ubuntu7)]
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 1562, in <module>
main()
File "/usr/bin/s3cmd", line 1542, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 1018, in cmd_sync
return cmd_sync_local2remote(args)
File "/usr/bin/s3cmd", line 987, in cmd_sync_local2remote
response = s3.object_put(src, uri, extra_headers, extra_label =
seq_label)
File "/usr/lib/python2.5/site-packages/S3/S3.py", line 202, in object_put
response = self.send_file(request, file, labels)
File "/usr/lib/python2.5/site-packages/S3/S3.py", line 498, in send_file
warning("Upload failed: %s (%s)" % (resource['uri'], S3Error(response)))
File "/usr/lib/python2.5/site-packages/S3/Exceptions.py", line 44, in
__init__
tree = getTreeFromXml(response["data"])
File "/usr/lib/python2.5/site-packages/S3/Utils.py", line 57, in
getTreeFromXml
tree = ET.fromstring(xml)
File "/usr/lib/python2.5/xml/etree/ElementTree.py", line 963, in XML
parser.feed(text)
File "/usr/lib/python2.5/xml/etree/ElementTree.py", line 1245, in feed
self._parser.Parse(data, 0)
ExpatError: mismatched tag: line 5, column 2
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the above lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
From: James K. <jk...@ai...> - 2013-04-08 20:17:48
|
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the following lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Problem: UnicodeDecodeErr: 'ascii' codec can't decode byte 0xcc in position 164: ordinal not in range(128)
S3cmd: 1.0.0
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 2006, in ?
main()
File "/usr/bin/s3cmd", line 1950, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 372, in cmd_object_put
exitcode, full_name, extra_headers["x-amz-meta-s3tools-gpgenc"] = gpg_encrypt(full_name_orig)
File "/usr/bin/s3cmd", line 1418, in gpg_encrypt
info(u"Encrypting file %(input_file)s to %(output_file)s..." % args)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position 164: ordinal not in range(128)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the above lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
James Kruger
Systems Administrator
American Institute of Biological Sciences
1900 Campus Commons Drive, Suite 200
Reston, VA 20191
Phone: 571-748-6155
Fax: 571-748-6155
E-mail: jk...@ai...
|
|
From: Leandro F. <le...@gm...> - 2013-04-12 21:22:11
|
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the following lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Problem: KeyError: 'elapsed'
S3cmd: 1.1.0-beta3
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 1800, in <module>
main()
File "/usr/bin/s3cmd", line 1741, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 309, in cmd_object_put
(unicodise(full_name_orig), uri_final, response["size"],
response["elapsed"],
KeyError: 'elapsed'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the above lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
--
Leandro Facchinetti
leafac.com
|
|
From: Michael W. <esi...@gm...> - 2013-04-13 06:30:08
|
This has been fixed in the latest 1.5.0 alpha versions. On 12 April 2013 23:21, Leandro Facchinetti <le...@gm...> wrote: > > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > An unexpected error has occurred. > Please report the following lines to: > s3t...@li... > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > Problem: KeyError: 'elapsed' > S3cmd: 1.1.0-beta3 > > Traceback (most recent call last): > File "/usr/bin/s3cmd", line 1800, in <module> > main() > File "/usr/bin/s3cmd", line 1741, in main > cmd_func(args) > File "/usr/bin/s3cmd", line 309, in cmd_object_put > (unicodise(full_name_orig), uri_final, response["size"], response["elapsed"], > KeyError: 'elapsed' > > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > An unexpected error has occurred. > Please report the above lines to: > s3t...@li... > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > > -- > Leandro Facchinetti > leafac.com > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > -- Michael Wood <esi...@gm...> |
|
From: Leandro F. <le...@gm...> - 2013-04-15 14:47:42
|
T hanks. On Sat, Apr 13, 2013 at 3:29 AM, Michael Wood <esi...@gm...> wrote: > This has been fixed in the latest 1.5.0 alpha versions. > > On 12 April 2013 23:21, Leandro Facchinetti <le...@gm...> wrote: > >> >> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> An unexpected error has occurred. >> Please report the following lines to: >> s3t...@li... >> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> >> Problem: KeyError: 'elapsed' >> S3cmd: 1.1.0-beta3 >> >> Traceback (most recent call last): >> File "/usr/bin/s3cmd", line 1800, in <module> >> main() >> File "/usr/bin/s3cmd", line 1741, in main >> cmd_func(args) >> File "/usr/bin/s3cmd", line 309, in cmd_object_put >> (unicodise(full_name_orig), uri_final, response["size"], response["elapsed"], >> KeyError: 'elapsed' >> >> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> An unexpected error has occurred. >> Please report the above lines to: >> s3t...@li... >> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> >> >> -- >> Leandro Facchinetti >> leafac.com >> >> >> ------------------------------------------------------------------------------ >> Precog is a next-generation analytics platform capable of advanced >> analytics on semi-structured data. The platform includes APIs for building >> apps and a phenomenal toolset for data science. Developers can use >> our toolset for easy data analysis & visualization. Get a free account! >> http://www2.precog.com/precogplatform/slashdotnewsletter >> > > > > -- > Michael Wood <esi...@gm...> -- Leandro Facchinetti leafac.com |
|
From: Michael B. <mic...@gm...> - 2013-08-18 17:14:33
|
hi there,
i got this error:
Problem: AttributeError: 'S3Error' object has no attribute 'Code'
S3cmd: 0.9.9.91
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 1736, in <module>
main()
File "/usr/bin/s3cmd", line 1681, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 44, in cmd_du
subcmd_bucket_usage(s3, uri)
File "/usr/bin/s3cmd", line 70, in subcmd_bucket_usage
if S3.codes.has_key(e.Code):
AttributeError: 'S3Error' object has no attribute 'Code'
while doing this:
s3cmd du s3://xxxxxxxxxxxxxx
my system: debian 6.0.6
uname -r: 2.6.32-5-xen-amd64
thanks for any help!
regards,
michael
|
|
From: Matt D. <ma...@do...> - 2013-08-18 23:57:56
|
You might try a much newer version, such as github.com/s3tools/s3cmd master branch. I've never seen this behavior, though I started with a later version. On Sun, Aug 18, 2013 at 12:14 PM, Michael Bladowski < mic...@gm...> wrote: > hi there, > > i got this error: > > Problem: AttributeError: 'S3Error' object has no attribute 'Code' > S3cmd: 0.9.9.91 > > Traceback (most recent call last): > File "/usr/bin/s3cmd", line 1736, in <module> > main() > File "/usr/bin/s3cmd", line 1681, in main > cmd_func(args) > File "/usr/bin/s3cmd", line 44, in cmd_du > subcmd_bucket_usage(s3, uri) > File "/usr/bin/s3cmd", line 70, in subcmd_bucket_usage > if S3.codes.has_key(e.Code): > AttributeError: 'S3Error' object has no attribute 'Code' > > while doing this: > > s3cmd du s3://xxxxxxxxxxxxxx > > my system: debian 6.0.6 > uname -r: 2.6.32-5-xen-amd64 > > thanks for any help! > > regards, > michael > > > > ------------------------------------------------------------------------------ > Get 100% visibility into Java/.NET code with AppDynamics Lite! > It's a free troubleshooting tool designed for production. > Get down to code-level detail for bottlenecks, with <2% overhead. > Download for free and get started troubleshooting in minutes. > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > > |
|
From: Bernhard W. <Ber...@ga...> - 2013-12-23 08:36:07
|
Problem: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 92: ordinal not in range(128)
S3cmd: 1.0.1
Traceback (most recent call last):
File "/usr/local/bin/s3cmd", line 2006, in <module>
main()
File "/usr/local/bin/s3cmd", line 1950, in main
cmd_func(args)
File "/usr/local/bin/s3cmd", line 1213, in cmd_sync
return cmd_sync_remote2local(args)
File "/usr/local/bin/s3cmd", line 966, in cmd_sync_remote2local
output(u"download: %s -> %s" % (remote_list[key]['object_uri_str'], remote_list[key]['local_filename']))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 92: ordinal not in range(128)
|
|
From: Chaz L. <cha...@gm...> - 2014-01-09 23:38:05
|
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An unexpected error has occurred. Please report the following lines to: s3t...@li... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Problem: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 152: ordinal not in range(128) S3cmd: 1.5.0-alpha1 Traceback (most recent call last): File "/usr/local/bin/s3cmd", line 2037, in <module> main() File "/usr/local/bin/s3cmd", line 1978, in main cmd_func(args) File "/usr/local/bin/s3cmd", line 1162, in cmd_sync return cmd_sync_local2remote(args) File "/usr/local/bin/s3cmd", line 1139, in cmd_sync_local2remote local_list, single_file_local = fetch_local_list(args[:-1], recursive = True) File "/usr/local/lib/python2.7/dist-packages/S3/FileLists.py", line 246, in fetch_local_list list_for_uri, single_file = _get_filelist_local(local_list, uri, cache) File "/usr/local/lib/python2.7/dist-packages/S3/FileLists.py", line 169, in _get_filelist_local for root, dirs, files in filelist: File "/usr/local/lib/python2.7/dist-packages/S3/FileLists.py", line 55, in _fswalk_no_symlinks handle_exclude_include_walk(dirpath, dirnames, filenames) File "/usr/local/lib/python2.7/dist-packages/S3/FileLists.py", line 137, in handle_exclude_include_walk debug(u"EXCLUDE: %s" % file) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 152: ordinal not in range(128) !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An unexpected error has occurred. Please report the above lines to: s3t...@li... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -- Chaz Larson Sent with Airmail |
|
From: Jonathan S. <sw...@gm...> - 2014-01-14 22:51:04
|
Hello,
I got the following error while running 's3cmd sync --dry-run
--skip-existing' on my Mac 10.8 using s3cmd version 1.0.1 installed from
Homebrew.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the following lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Problem: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in
position 48: ordinal not in range(128)
S3cmd: 1.0.1
Traceback (most recent call last):
File "/usr/local/bin/s3cmd", line 2006, in <module>
main()
File "/usr/local/bin/s3cmd", line 1950, in main
cmd_func(args)
File "/usr/local/bin/s3cmd", line 1213, in cmd_sync
return cmd_sync_remote2local(args)
File "/usr/local/bin/s3cmd", line 966, in cmd_sync_remote2local
output(u"download: %s -> %s" % (remote_list[key]['object_uri_str'],
remote_list[key]['local_filename']))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 48:
ordinal not in range(128)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the above lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
From: Florian A. <flo...@wy...> - 2014-03-05 15:41:21
|
Hi,
this occurred when I used a bucket name with a typo on CentOS 5.8:
# s3cmd du s3://mybucket
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the following lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Problem: AttributeErr: S3Error instance has no attribute 'Code'
S3cmd: 0.9.9.91
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 1736, in ?
main()
File "/usr/bin/s3cmd", line 1681, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 44, in cmd_du
subcmd_bucket_usage(s3, uri)
File "/usr/bin/s3cmd", line 70, in subcmd_bucket_usage
if S3.codes.has_key(e.Code):
AttributeError: S3Error instance has no attribute 'Code'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the above lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
From: Matt D. <ma...@do...> - 2014-03-05 15:54:57
|
The version you have is very very (many years) old. Please repeat with a newer version such as github.com/s3tools/s3cmd master branch and report success/failure. On Wed, Mar 5, 2014 at 7:10 AM, Florian Anderiasch < flo...@wy...> wrote: > Hi, > this occurred when I used a bucket name with a typo on CentOS 5.8: > > # s3cmd du s3://mybucket > > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > An unexpected error has occurred. > Please report the following lines to: > s3t...@li... > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > Problem: AttributeErr: S3Error instance has no attribute 'Code' > S3cmd: 0.9.9.91 > > Traceback (most recent call last): > File "/usr/bin/s3cmd", line 1736, in ? > main() > File "/usr/bin/s3cmd", line 1681, in main > cmd_func(args) > File "/usr/bin/s3cmd", line 44, in cmd_du > subcmd_bucket_usage(s3, uri) > File "/usr/bin/s3cmd", line 70, in subcmd_bucket_usage > if S3.codes.has_key(e.Code): > AttributeError: S3Error instance has no attribute 'Code' > > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > An unexpected error has occurred. > Please report the above lines to: > s3t...@li... > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > > ------------------------------------------------------------------------------ > Subversion Kills Productivity. Get off Subversion & Make the Move to > Perforce. > With Perforce, you get hassle-free workflows. Merge that actually works. > Faster operations. Version large binaries. Built-in WAN optimization and > the > freedom to use Git, Perforce or both. Make the move to Perforce. > > http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk > > |
|
From: Matt D. <ma...@do...> - 2014-03-05 15:56:05
|
You will also need to follow these directions for Python 2.4 support at the moment: > Get this file and drop it into your S3/ directory. > https://gc3pie.googlecode.com/svn-history/r1790/trunk/gc3pie/gc3libs/compat/collections.py > > or find another implementation that provides it. On Wed, Mar 5, 2014 at 7:54 AM, Matt Domsch <ma...@do...> wrote: > The version you have is very very (many years) old. Please repeat with a > newer version such as github.com/s3tools/s3cmd master branch and report > success/failure. > > > On Wed, Mar 5, 2014 at 7:10 AM, Florian Anderiasch < > flo...@wy...> wrote: > >> Hi, >> this occurred when I used a bucket name with a typo on CentOS 5.8: >> >> # s3cmd du s3://mybucket >> >> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> An unexpected error has occurred. >> Please report the following lines to: >> s3t...@li... >> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> >> Problem: AttributeErr: S3Error instance has no attribute 'Code' >> S3cmd: 0.9.9.91 >> >> Traceback (most recent call last): >> File "/usr/bin/s3cmd", line 1736, in ? >> main() >> File "/usr/bin/s3cmd", line 1681, in main >> cmd_func(args) >> File "/usr/bin/s3cmd", line 44, in cmd_du >> subcmd_bucket_usage(s3, uri) >> File "/usr/bin/s3cmd", line 70, in subcmd_bucket_usage >> if S3.codes.has_key(e.Code): >> AttributeError: S3Error instance has no attribute 'Code' >> >> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> An unexpected error has occurred. >> Please report the above lines to: >> s3t...@li... >> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> >> >> ------------------------------------------------------------------------------ >> Subversion Kills Productivity. Get off Subversion & Make the Move to >> Perforce. >> With Perforce, you get hassle-free workflows. Merge that actually works. >> Faster operations. Version large binaries. Built-in WAN optimization and >> the >> freedom to use Git, Perforce or both. Make the move to Perforce. >> >> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk >> >> > |
|
From: Ivan <upr...@gm...> - 2014-03-29 08:20:52
|
Hello, I saw that report when I tried to sync my sites with S3. Total data
size - about 250GB, and a lot of small files.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please try reproducing the error using
the latest s3cmd code from the git master
branch found at:
https://github.com/s3tools/s3cmd
If the error persists, please report the
following lines (removing any private
info as necessary) to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Invoked as: /usr/bin/s3cmd sync -v www/ s3://blackhost/bunker/Problem:
KeyError: 'dev'
S3cmd: 1.5.0-beta1
python: 2.6.6 (r266:84292, Jan 22 2014, 09:42:36)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)]
environment LANG=ru_RU.UTF-8
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 2422, in <module>
main()
File "/usr/bin/s3cmd", line 2344, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 1414, in cmd_sync
return cmd_sync_local2remote(args)
File "/usr/bin/s3cmd", line 1398, in cmd_sync_local2remote
destination_base_uri = _single_process(local_list)
File "/usr/bin/s3cmd", line 1209, in _single_process
_child(destination_base, local_list)
File "/usr/bin/s3cmd", line 1295, in _child
local_list, remote_list, update_list, copy_pairs =
compare_filelists(local_list, remote_list, src_remote = False, dst_remote =
True, delay_updates = cfg.delay_updates)
File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 563, in
compare_filelists
md5 = src_list.get_md5(relative_file)
File "/usr/lib/python2.6/site-packages/S3/FileDict.py", line 40, in
get_md5
md5 = self.get_hardlink_md5(relative_file)
File "/usr/lib/python2.6/site-packages/S3/FileDict.py", line 60, in
get_hardlink_md5
dev = self[relative_file]['dev']
KeyError: 'dev'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please try reproducing the error using
the latest s3cmd code from the git master
branch found at:
https://github.com/s3tools/s3cmd
If the error persists, please report the
above lines (removing any private
info as necessary) to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
From: Matt D. <ma...@do...> - 2014-03-29 13:13:12
|
Ivan, thanks for the report. I just pushed this change to the master
branch which should fix it:
commit 0b3deec34dabe6d34b1034620aa18096504d3c92
Author: Matt Domsch <ma...@do...>
Date: Sat Mar 29 08:11:57 2014 -0500
hardlink fix
If we didn't record the hardlink md5 because the file size
was zero, don't then fail to look it up later.
diff --git a/S3/FileDict.py b/S3/FileDict.py
index 2c21767..4f2ab6f 100644
--- a/S3/FileDict.py
+++ b/S3/FileDict.py
@@ -57,10 +57,10 @@ class FileDict(SortedDict):
def get_hardlink_md5(self, relative_file):
md5 = None
- dev = self[relative_file]['dev']
- inode = self[relative_file]['inode']
try:
+ dev = self[relative_file]['dev']
+ inode = self[relative_file]['inode']
md5 = self.hardlinks[dev][inode]['md5']
- except:
+ except KeyError:
pass
return md5
On Sat, Mar 29, 2014 at 3:20 AM, Ivan <upr...@gm...> wrote:
> Hello, I saw that report when I tried to sync my sites with S3. Total data
> size - about 250GB, and a lot of small files.
>
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> An unexpected error has occurred.
> Please try reproducing the error using
> the latest s3cmd code from the git master
> branch found at:
> https://github.com/s3tools/s3cmd
> If the error persists, please report the
> following lines (removing any private
> info as necessary) to:
> s3t...@li...
>
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>
> Invoked as: /usr/bin/s3cmd sync -v www/ s3://blackhost/bunker/Problem:
> KeyError: 'dev'
> S3cmd: 1.5.0-beta1
> python: 2.6.6 (r266:84292, Jan 22 2014, 09:42:36)
> [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)]
> environment LANG=ru_RU.UTF-8
>
> Traceback (most recent call last):
> File "/usr/bin/s3cmd", line 2422, in <module>
> main()
> File "/usr/bin/s3cmd", line 2344, in main
> cmd_func(args)
> File "/usr/bin/s3cmd", line 1414, in cmd_sync
> return cmd_sync_local2remote(args)
> File "/usr/bin/s3cmd", line 1398, in cmd_sync_local2remote
> destination_base_uri = _single_process(local_list)
> File "/usr/bin/s3cmd", line 1209, in _single_process
> _child(destination_base, local_list)
> File "/usr/bin/s3cmd", line 1295, in _child
> local_list, remote_list, update_list, copy_pairs =
> compare_filelists(local_list, remote_list, src_remote = False, dst_remote =
> True, delay_updates = cfg.delay_updates)
> File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 563, in
> compare_filelists
> md5 = src_list.get_md5(relative_file)
> File "/usr/lib/python2.6/site-packages/S3/FileDict.py", line 40, in
> get_md5
> md5 = self.get_hardlink_md5(relative_file)
> File "/usr/lib/python2.6/site-packages/S3/FileDict.py", line 60, in
> get_hardlink_md5
> dev = self[relative_file]['dev']
> KeyError: 'dev'
>
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> An unexpected error has occurred.
> Please try reproducing the error using
> the latest s3cmd code from the git master
> branch found at:
> https://github.com/s3tools/s3cmd
> If the error persists, please report the
> above lines (removing any private
> info as necessary) to:
> s3t...@li...
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>
>
> ------------------------------------------------------------------------------
>
>
|
|
From: Shawn C. <sh...@do...> - 2014-04-05 13:29:45
|
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Problem: UnicodeEncodeError: 'ascii' codec can't encode characters in
position 83-84: ordinal not in range(128)
S3cmd: 1.5.0-alpha3
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 2094, in <module>
main()
File "/usr/bin/s3cmd", line 2034, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 274, in cmd_object_put
local_list, single_file_local = fetch_local_list(args)
File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 240, in
fetch_local_list
if uri.isdir() and not recursive:
File "/usr/lib/python2.6/site-packages/S3/S3Uri.py", line 158, in isdir
return os.path.isdir(self.path())
File "/usr/lib64/python2.6/genericpath.py", line 41, in isdir
st = os.stat(s)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
83-84: ordinal not in range(128)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the above lines to:
s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
From: Matt D. <ma...@do...> - 2014-04-05 17:57:02
|
I think this was fixed this last week in upstream github.com/s3tools/s3cmd master branch - specifically, uploading content using 'put' where the filename is UTF-8-encoded unicode. Thanks, Matt On Sat, Apr 5, 2014 at 8:06 AM, Shawn Crowe <sh...@do...> wrote: > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > Problem: UnicodeEncodeError: 'ascii' codec can't encode characters in > position 83-84: ordinal not in range(128) > S3cmd: 1.5.0-alpha3 > > Traceback (most recent call last): > File "/usr/bin/s3cmd", line 2094, in <module> > main() > File "/usr/bin/s3cmd", line 2034, in main > cmd_func(args) > File "/usr/bin/s3cmd", line 274, in cmd_object_put > local_list, single_file_local = fetch_local_list(args) > File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 240, in > fetch_local_list > if uri.isdir() and not recursive: > File "/usr/lib/python2.6/site-packages/S3/S3Uri.py", line 158, in isdir > return os.path.isdir(self.path()) > File "/usr/lib64/python2.6/genericpath.py", line 41, in isdir > st = os.stat(s) > UnicodeEncodeError: 'ascii' codec can't encode characters in position 83-84: > ordinal not in range(128) > > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > An unexpected error has occurred. > Please report the above lines to: > s3t...@li... > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > ------------------------------------------------------------------------------ > |
|
From: Shawn C. <sh...@do...> - 2014-04-11 02:32:09
|
I'm still seeing some errors after updating..
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please try reproducing the error using
the latest s3cmd code from the git master
branch found at:
https://github.com/s3tools/s3cmd
If the error persists, please report the
following lines (removing any private
info as necessary) to:
s3t...@li...
You have encountered a UnicodeEncodeError. Your environment
variable LANG=C may not specify a Unicode encoding (e.g. UTF-8).
Please set LANG=en_US.UTF-8 or similar in your environment before
invoking s3cmd.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Invoked as: /usr/bin/s3cmd put --config=/source/.s3cfg --no-check-md5
--skip-existing --no-encrypt -P -H -p --add-header=Expires:Fri, 10 Apr 2015
13:47:49 GMT --add-header=Cache-Control:max-age=31536000, public
/source/imgs/2014/04/397x200_1397137668_Taiw??n.JPG
s3://bucket/destinationpath/
Environment variable is:
LANG=en_US.UTF-8
Original filename is:
397x200_1397137668_Taiwán.JPG
Any suggestions?
Thanks,
Shawn
On Sat, Apr 5, 2014 at 11:56 AM, Matt Domsch <ma...@do...> wrote:
> I think this was fixed this last week in upstream
> github.com/s3tools/s3cmd master branch - specifically, uploading
> content using 'put' where the filename is UTF-8-encoded unicode.
>
> Thanks,
> Matt
>
> On Sat, Apr 5, 2014 at 8:06 AM, Shawn Crowe <sh...@do...> wrote:
> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> >
> > Problem: UnicodeEncodeError: 'ascii' codec can't encode characters in
> > position 83-84: ordinal not in range(128)
> > S3cmd: 1.5.0-alpha3
> >
> > Traceback (most recent call last):
> > File "/usr/bin/s3cmd", line 2094, in <module>
> > main()
> > File "/usr/bin/s3cmd", line 2034, in main
> > cmd_func(args)
> > File "/usr/bin/s3cmd", line 274, in cmd_object_put
> > local_list, single_file_local = fetch_local_list(args)
> > File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 240, in
> > fetch_local_list
> > if uri.isdir() and not recursive:
> > File "/usr/lib/python2.6/site-packages/S3/S3Uri.py", line 158, in isdir
> > return os.path.isdir(self.path())
> > File "/usr/lib64/python2.6/genericpath.py", line 41, in isdir
> > st = os.stat(s)
> > UnicodeEncodeError: 'ascii' codec can't encode characters in position
> 83-84:
> > ordinal not in range(128)
> >
> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> > An unexpected error has occurred.
> > Please report the above lines to:
> > s3t...@li...
> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> >
> >
> ------------------------------------------------------------------------------
> >
>
|
|
From: Matt D. <ma...@do...> - 2014-04-11 03:21:05
|
Can you send the rest of the error report? There should have been a traceback included. I can't reproduce this on Fedora 20 (python 2.7) or RHEL 5 (python 2.4). What OS version and python version do you have please? Thanks, Matt On Thu, Apr 10, 2014 at 9:00 PM, Shawn Crowe <sh...@do...> wrote: > I'm still seeing some errors after updating.. > > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > An unexpected error has occurred. > Please try reproducing the error using > the latest s3cmd code from the git master > branch found at: > https://github.com/s3tools/s3cmd > If the error persists, please report the > following lines (removing any private > info as necessary) to: > s3t...@li... > > You have encountered a UnicodeEncodeError. Your environment > variable LANG=C may not specify a Unicode encoding (e.g. UTF-8). > Please set LANG=en_US.UTF-8 or similar in your environment before > invoking s3cmd. > > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > Invoked as: /usr/bin/s3cmd put --config=/source/.s3cfg --no-check-md5 > --skip-existing --no-encrypt -P -H -p --add-header=Expires:Fri, 10 Apr 2015 > 13:47:49 GMT --add-header=Cache-Control:max-age=31536000, public > /source/imgs/2014/04/397x200_1397137668_Taiw??n.JPG > s3://bucket/destinationpath/ > > Environment variable is: > LANG=en_US.UTF-8 > > Original filename is: > 397x200_1397137668_Taiwán.JPG > > Any suggestions? > > Thanks, > Shawn > > On Sat, Apr 5, 2014 at 11:56 AM, Matt Domsch <ma...@do...> wrote: > >> I think this was fixed this last week in upstream >> github.com/s3tools/s3cmd master branch - specifically, uploading >> content using 'put' where the filename is UTF-8-encoded unicode. >> >> Thanks, >> Matt >> >> On Sat, Apr 5, 2014 at 8:06 AM, Shawn Crowe <sh...@do...> wrote: >> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> > >> > Problem: UnicodeEncodeError: 'ascii' codec can't encode characters in >> > position 83-84: ordinal not in range(128) >> > S3cmd: 1.5.0-alpha3 >> > >> > Traceback (most recent call last): >> > File "/usr/bin/s3cmd", line 2094, in <module> >> > main() >> > File "/usr/bin/s3cmd", line 2034, in main >> > cmd_func(args) >> > File "/usr/bin/s3cmd", line 274, in cmd_object_put >> > local_list, single_file_local = fetch_local_list(args) >> > File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 240, in >> > fetch_local_list >> > if uri.isdir() and not recursive: >> > File "/usr/lib/python2.6/site-packages/S3/S3Uri.py", line 158, in >> isdir >> > return os.path.isdir(self.path()) >> > File "/usr/lib64/python2.6/genericpath.py", line 41, in isdir >> > st = os.stat(s) >> > UnicodeEncodeError: 'ascii' codec can't encode characters in position >> 83-84: >> > ordinal not in range(128) >> > >> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> > An unexpected error has occurred. >> > Please report the above lines to: >> > s3t...@li... >> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >> > >> > >> ------------------------------------------------------------------------------ >> > >> > > |
|
From: Shawn C. <sh...@do...> - 2014-04-11 10:35:50
|
Invoked as: /usr/bin/s3cmd put --config=/var/www/html/.s3cfg --no-check-md5
--skip-existing --no-encrypt -P -H -p --add-header=Expires:Fri, 10 Apr 2015
13:47:49 GMT --add-header=Cache-Control:max-age=31536000, public
/source/imgs/2014/04/397x200_1397137668_Taiw??n.JPG
s3://bucket/destinationpath/
Problem: UnicodeEncodeError: 'ascii' codec can't encode characters in
position 59-60: ordinal not in range(128)
S3cmd: 1.5.0-beta1
python: 2.6.8 (unknown, Mar 14 2013, 09:31:22)
[GCC 4.6.2 20111027 (Red Hat 4.6.2-2)]
environment LANG=C
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 2422, in <module>
main()
File "/usr/bin/s3cmd", line 2344, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 306, in cmd_object_put
local_list, single_file_local, exclude_list = fetch_local_list(args,
is_src = True)
File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 301, in
fetch_local_list
if uri.isdir() and not recursive:
File "/usr/lib/python2.6/site-packages/S3/S3Uri.py", line 159, in isdir
return os.path.isdir(self.path())
File "/usr/lib64/python2.6/genericpath.py", line 41, in isdir
st = os.stat(s)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
59-60: ordinal not in range(128)
On Thu, Apr 10, 2014 at 9:20 PM, Matt Domsch <ma...@do...> wrote:
> Can you send the rest of the error report? There should have been a
> traceback included. I can't reproduce this on Fedora 20 (python 2.7) or
> RHEL 5 (python 2.4). What OS version and python version do you have please?
>
> Thanks,
> Matt
>
>
>
> On Thu, Apr 10, 2014 at 9:00 PM, Shawn Crowe <sh...@do...> wrote:
>
>> I'm still seeing some errors after updating..
>>
>> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>> An unexpected error has occurred.
>> Please try reproducing the error using
>> the latest s3cmd code from the git master
>> branch found at:
>> https://github.com/s3tools/s3cmd
>> If the error persists, please report the
>> following lines (removing any private
>> info as necessary) to:
>> s3t...@li...
>>
>> You have encountered a UnicodeEncodeError. Your environment
>> variable LANG=C may not specify a Unicode encoding (e.g. UTF-8).
>> Please set LANG=en_US.UTF-8 or similar in your environment before
>> invoking s3cmd.
>>
>> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>
>> Invoked as: /usr/bin/s3cmd put --config=/source/.s3cfg --no-check-md5
>> --skip-existing --no-encrypt -P -H -p --add-header=Expires:Fri, 10 Apr 2015
>> 13:47:49 GMT --add-header=Cache-Control:max-age=31536000, public
>> /source/imgs/2014/04/397x200_1397137668_Taiw??n.JPG
>> s3://bucket/destinationpath/
>>
>> Environment variable is:
>> LANG=en_US.UTF-8
>>
>> Original filename is:
>> 397x200_1397137668_Taiwán.JPG
>>
>> Any suggestions?
>>
>> Thanks,
>> Shawn
>>
>> On Sat, Apr 5, 2014 at 11:56 AM, Matt Domsch <ma...@do...> wrote:
>>
>>> I think this was fixed this last week in upstream
>>> github.com/s3tools/s3cmd master branch - specifically, uploading
>>> content using 'put' where the filename is UTF-8-encoded unicode.
>>>
>>> Thanks,
>>> Matt
>>>
>>> On Sat, Apr 5, 2014 at 8:06 AM, Shawn Crowe <sh...@do...> wrote:
>>> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>> >
>>> > Problem: UnicodeEncodeError: 'ascii' codec can't encode characters in
>>> > position 83-84: ordinal not in range(128)
>>> > S3cmd: 1.5.0-alpha3
>>> >
>>> > Traceback (most recent call last):
>>> > File "/usr/bin/s3cmd", line 2094, in <module>
>>> > main()
>>> > File "/usr/bin/s3cmd", line 2034, in main
>>> > cmd_func(args)
>>> > File "/usr/bin/s3cmd", line 274, in cmd_object_put
>>> > local_list, single_file_local = fetch_local_list(args)
>>> > File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 240, in
>>> > fetch_local_list
>>> > if uri.isdir() and not recursive:
>>> > File "/usr/lib/python2.6/site-packages/S3/S3Uri.py", line 158, in
>>> isdir
>>> > return os.path.isdir(self.path())
>>> > File "/usr/lib64/python2.6/genericpath.py", line 41, in isdir
>>> > st = os.stat(s)
>>> > UnicodeEncodeError: 'ascii' codec can't encode characters in position
>>> 83-84:
>>> > ordinal not in range(128)
>>> >
>>> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>> > An unexpected error has occurred.
>>> > Please report the above lines to:
>>> > s3t...@li...
>>> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> >
>>>
>>
>>
>
|
|
From: Matt D. <ma...@do...> - 2014-04-11 11:44:05
|
Please try: export LANG=en_US.UTF-8 /usr/sbin/s3cmd put ... I get the same failure if my environment has LANG=C (as yours does), and no failure if I set LANG=en_US.UTF-8. This setting changes the behavior of the python on.stat() call, which is where the failure is occurring. Thanks, Matt On Fri, Apr 11, 2014 at 5:35 AM, Shawn Crowe <sh...@do...> wrote: > Invoked as: /usr/bin/s3cmd put --config=/var/www/html/.s3cfg > --no-check-md5 --skip-existing --no-encrypt -P -H -p > --add-header=Expires:Fri, 10 Apr 2015 13:47:49 GMT > --add-header=Cache-Control:max-age=31536000, public > /source/imgs/2014/04/397x200_1397137668_Taiw??n.JPG > s3://bucket/destinationpath/ > Problem: UnicodeEncodeError: 'ascii' codec can't encode characters in > position 59-60: ordinal not in range(128) > S3cmd: 1.5.0-beta1 > python: 2.6.8 (unknown, Mar 14 2013, 09:31:22) > [GCC 4.6.2 20111027 (Red Hat 4.6.2-2)] > environment LANG=C > > Traceback (most recent call last): > File "/usr/bin/s3cmd", line 2422, in <module> > main() > File "/usr/bin/s3cmd", line 2344, in main > cmd_func(args) > File "/usr/bin/s3cmd", line 306, in cmd_object_put > local_list, single_file_local, exclude_list = fetch_local_list(args, > is_src = True) > File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 301, in > fetch_local_list > if uri.isdir() and not recursive: > File "/usr/lib/python2.6/site-packages/S3/S3Uri.py", line 159, in isdir > return os.path.isdir(self.path()) > File "/usr/lib64/python2.6/genericpath.py", line 41, in isdir > st = os.stat(s) > UnicodeEncodeError: 'ascii' codec can't encode characters in position > 59-60: ordinal not in range(128) > > > > On Thu, Apr 10, 2014 at 9:20 PM, Matt Domsch <ma...@do...> wrote: > >> Can you send the rest of the error report? There should have been a >> traceback included. I can't reproduce this on Fedora 20 (python 2.7) or >> RHEL 5 (python 2.4). What OS version and python version do you have please? >> >> Thanks, >> Matt >> >> >> >> On Thu, Apr 10, 2014 at 9:00 PM, Shawn Crowe <sh...@do...> wrote: >> >>> I'm still seeing some errors after updating.. >>> >>> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >>> An unexpected error has occurred. >>> Please try reproducing the error using >>> the latest s3cmd code from the git master >>> branch found at: >>> https://github.com/s3tools/s3cmd >>> If the error persists, please report the >>> following lines (removing any private >>> info as necessary) to: >>> s3t...@li... >>> >>> You have encountered a UnicodeEncodeError. Your environment >>> variable LANG=C may not specify a Unicode encoding (e.g. UTF-8). >>> Please set LANG=en_US.UTF-8 or similar in your environment before >>> invoking s3cmd. >>> >>> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >>> >>> Invoked as: /usr/bin/s3cmd put --config=/source/.s3cfg --no-check-md5 >>> --skip-existing --no-encrypt -P -H -p --add-header=Expires:Fri, 10 Apr 2015 >>> 13:47:49 GMT --add-header=Cache-Control:max-age=31536000, public >>> /source/imgs/2014/04/397x200_1397137668_Taiw??n.JPG >>> s3://bucket/destinationpath/ >>> >>> Environment variable is: >>> LANG=en_US.UTF-8 >>> >>> Original filename is: >>> 397x200_1397137668_Taiwán.JPG >>> >>> Any suggestions? >>> >>> Thanks, >>> Shawn >>> >>> On Sat, Apr 5, 2014 at 11:56 AM, Matt Domsch <ma...@do...> wrote: >>> >>>> I think this was fixed this last week in upstream >>>> github.com/s3tools/s3cmd master branch - specifically, uploading >>>> content using 'put' where the filename is UTF-8-encoded unicode. >>>> >>>> Thanks, >>>> Matt >>>> >>>> On Sat, Apr 5, 2014 at 8:06 AM, Shawn Crowe <sh...@do...> wrote: >>>> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >>>> > >>>> > Problem: UnicodeEncodeError: 'ascii' codec can't encode characters in >>>> > position 83-84: ordinal not in range(128) >>>> > S3cmd: 1.5.0-alpha3 >>>> > >>>> > Traceback (most recent call last): >>>> > File "/usr/bin/s3cmd", line 2094, in <module> >>>> > main() >>>> > File "/usr/bin/s3cmd", line 2034, in main >>>> > cmd_func(args) >>>> > File "/usr/bin/s3cmd", line 274, in cmd_object_put >>>> > local_list, single_file_local = fetch_local_list(args) >>>> > File "/usr/lib/python2.6/site-packages/S3/FileLists.py", line 240, >>>> in >>>> > fetch_local_list >>>> > if uri.isdir() and not recursive: >>>> > File "/usr/lib/python2.6/site-packages/S3/S3Uri.py", line 158, in >>>> isdir >>>> > return os.path.isdir(self.path()) >>>> > File "/usr/lib64/python2.6/genericpath.py", line 41, in isdir >>>> > st = os.stat(s) >>>> > UnicodeEncodeError: 'ascii' codec can't encode characters in position >>>> 83-84: >>>> > ordinal not in range(128) >>>> > >>>> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >>>> > An unexpected error has occurred. >>>> > Please report the above lines to: >>>> > s3t...@li... >>>> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! >>>> > >>>> > >>>> ------------------------------------------------------------------------------ >>>> > >>>> >>> >>> >> > |
|
From: Alexander S. <as...@ne...> - 2014-05-14 12:26:25
|
Hello!
I executed the following command:
s3cmd ls s3://some-bucket-backups/Backup/SCTP/AppData/ | more
I encountered the following bug:
Problem: IOError: [Errno 32] Broken pipe
S3cmd: 1.0.0
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 2006, in <module>
main()
File "/usr/bin/s3cmd", line 1950, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 125, in cmd_ls
subcmd_bucket_list(s3, uri)
File "/usr/bin/s3cmd", line 183, in subcmd_bucket_list
"uri": uri.compose_uri(bucket, object["Key"]),
File "/usr/bin/s3cmd", line 33, in output
sys.stdout.write(message + "\n")
File "/usr/lib64/python2.6/codecs.py", line 352, in write
self.stream.write(data)
IOError: [Errno 32] Broken pipe
I found that I got 24736 as the result of the command
s3cmd ls s3://some-bucket-backups/Backup/SCTP/AppData/ | grep -c .
while the real number of files was varying. In the end, the number of files
would be about 38000.
I use the name s3://some-bucket-backups of the bucket to hide the real name
of the backet, which I don't want to disclose.
Thanks.
|
|
From: Matt D. <ma...@do...> - 2014-05-14 13:54:24
|
This is fixed in upstream github.com/s3tools/s3cmd master branch. It failed to write to stdout after grep closed its stdin, thus tracebacked. We no longer treat broken pipes as fatal in the upstream master branch. On Wed, May 14, 2014 at 6:58 AM, Alexander Semigrodskikh < as...@ne...> wrote: > Hello! > > I executed the following command: > > s3cmd ls s3://some-bucket-backups/Backup/SCTP/AppData/ | more > > > I encountered the following bug: > > Problem: IOError: [Errno 32] Broken pipe > S3cmd: 1.0.0 > > Traceback (most recent call last): > File "/usr/bin/s3cmd", line 2006, in <module> > main() > File "/usr/bin/s3cmd", line 1950, in main > cmd_func(args) > File "/usr/bin/s3cmd", line 125, in cmd_ls > subcmd_bucket_list(s3, uri) > File "/usr/bin/s3cmd", line 183, in subcmd_bucket_list > "uri": uri.compose_uri(bucket, object["Key"]), > File "/usr/bin/s3cmd", line 33, in output > sys.stdout.write(message + "\n") > File "/usr/lib64/python2.6/codecs.py", line 352, in write > self.stream.write(data) > IOError: [Errno 32] Broken pipe > > I found that I got 24736 as the result of the command > > s3cmd ls s3://some-bucket-backups/Backup/SCTP/AppData/ | grep -c . > > while the real number of files was varying. In the end, the number of > files would be about 38000. > > I use the name s3://some-bucket-backups of the bucket to hide the real > name of the backet, which I don't want to disclose. > > Thanks. > > > ------------------------------------------------------------------------------ > "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE > Instantly run your Selenium tests across 300+ browser/OS combos. > Get unparalleled scalability from the best Selenium testing platform > available > Simple to use. Nothing to install. Get started now for free." > http://p.sf.net/sfu/SauceLabs > |
|
From: Alexander S. <as...@ne...> - 2014-05-14 14:12:47
|
Thanks for your response! Is the "limit" of 24736 related to this stdout problem? With kind regards, Alexander Semigrodskikh On Wed, May 14, 2014 at 7:54 PM, Matt Domsch <ma...@do...> wrote: > This is fixed in upstream github.com/s3tools/s3cmd master branch. It > failed to write to stdout after grep closed its stdin, thus tracebacked. > We no longer treat broken pipes as fatal in the upstream master branch. > > > On Wed, May 14, 2014 at 6:58 AM, Alexander Semigrodskikh < > as...@ne...> wrote: > >> Hello! >> >> I executed the following command: >> >> s3cmd ls s3://some-bucket-backups/Backup/SCTP/AppData/ | more >> >> >> I encountered the following bug: >> >> Problem: IOError: [Errno 32] Broken pipe >> S3cmd: 1.0.0 >> >> Traceback (most recent call last): >> File "/usr/bin/s3cmd", line 2006, in <module> >> main() >> File "/usr/bin/s3cmd", line 1950, in main >> cmd_func(args) >> File "/usr/bin/s3cmd", line 125, in cmd_ls >> subcmd_bucket_list(s3, uri) >> File "/usr/bin/s3cmd", line 183, in subcmd_bucket_list >> "uri": uri.compose_uri(bucket, object["Key"]), >> File "/usr/bin/s3cmd", line 33, in output >> sys.stdout.write(message + "\n") >> File "/usr/lib64/python2.6/codecs.py", line 352, in write >> self.stream.write(data) >> IOError: [Errno 32] Broken pipe >> >> I found that I got 24736 as the result of the command >> >> s3cmd ls s3://some-bucket-backups/Backup/SCTP/AppData/ | grep -c . >> >> while the real number of files was varying. In the end, the number of >> files would be about 38000. >> >> I use the name s3://some-bucket-backups of the bucket to hide the real >> name of the backet, which I don't want to disclose. >> >> Thanks. >> >> >> ------------------------------------------------------------------------------ >> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE >> Instantly run your Selenium tests across 300+ browser/OS combos. >> Get unparalleled scalability from the best Selenium testing platform >> available >> Simple to use. Nothing to install. Get started now for free." >> http://p.sf.net/sfu/SauceLabs >> > > |
|
From: Matt D. <ma...@do...> - 2014-05-14 15:18:56
|
Unclear - there aren't any "limits" intentionally in the code. I am successfully using it for nearly 1M files, and others have used it well above that. On Wed, May 14, 2014 at 9:12 AM, Alexander Semigrodskikh < as...@ne...> wrote: > Thanks for your response! Is the "limit" of 24736 related to this stdout > problem? > > With kind regards, > Alexander Semigrodskikh > > > On Wed, May 14, 2014 at 7:54 PM, Matt Domsch <ma...@do...> wrote: > >> This is fixed in upstream github.com/s3tools/s3cmd master branch. It >> failed to write to stdout after grep closed its stdin, thus tracebacked. >> We no longer treat broken pipes as fatal in the upstream master branch. >> >> >> On Wed, May 14, 2014 at 6:58 AM, Alexander Semigrodskikh < >> as...@ne...> wrote: >> >>> Hello! >>> >>> I executed the following command: >>> >>> s3cmd ls s3://some-bucket-backups/Backup/SCTP/AppData/ | more >>> >>> >>> I encountered the following bug: >>> >>> Problem: IOError: [Errno 32] Broken pipe >>> S3cmd: 1.0.0 >>> >>> Traceback (most recent call last): >>> File "/usr/bin/s3cmd", line 2006, in <module> >>> main() >>> File "/usr/bin/s3cmd", line 1950, in main >>> cmd_func(args) >>> File "/usr/bin/s3cmd", line 125, in cmd_ls >>> subcmd_bucket_list(s3, uri) >>> File "/usr/bin/s3cmd", line 183, in subcmd_bucket_list >>> "uri": uri.compose_uri(bucket, object["Key"]), >>> File "/usr/bin/s3cmd", line 33, in output >>> sys.stdout.write(message + "\n") >>> File "/usr/lib64/python2.6/codecs.py", line 352, in write >>> self.stream.write(data) >>> IOError: [Errno 32] Broken pipe >>> >>> I found that I got 24736 as the result of the command >>> >>> s3cmd ls s3://some-bucket-backups/Backup/SCTP/AppData/ | grep -c . >>> >>> while the real number of files was varying. In the end, the number of >>> files would be about 38000. >>> >>> I use the name s3://some-bucket-backups of the bucket to hide the real >>> name of the backet, which I don't want to disclose. >>> >>> Thanks. >>> >>> >>> ------------------------------------------------------------------------------ >>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE >>> Instantly run your Selenium tests across 300+ browser/OS combos. >>> Get unparalleled scalability from the best Selenium testing platform >>> available >>> Simple to use. Nothing to install. Get started now for free." >>> http://p.sf.net/sfu/SauceLabs >>> >> >> > |
Thanks for helping keep SourceForge clean.
X