|
From: William A. <al...@ui...> - 2019-06-19 15:48:50
|
I am using s3cmd on a Windows machine and trying to download a file from the the Amazon s3 bucket to a local network drive location. The AWS account is provided by a 3rd party CRM company. I do not believe there are permissions issues at play. I am able to ls for listing out directories and files. I have successfully performed this download using the aws cli, but am trying s3cmd as an alternative. I am wanting to build a batch file for multiple file downloads and uploads to our s3 buckets. This is part of a standing daily run, so your help greatly appreciated. I have used the -d debug flag to in running the cp command to provide output for you. I've x-d out our Amazon s3 username within the s3 bucket file reference. The Command Prompt output follows: c:\Users\albinw\Documents\s3cmd-2.0.2>python s3cmd -d cp s3://radius-crm-s3-us/xxxxxxxxxxxxxxxx-1469026373674/Exports/DST_InquiriesList_May/DST_Inquiries_List-May_2019.csv H://Documents/GAWG_R-Data_Folder/DST_Inquiries_List-May_2019.csv >> H://Documents/GAWG_R-Data_Folder/GAWG_Results.txt DEBUG: s3cmd version 2.0.2 DEBUG: ConfigParser: Reading file 'C:\Users\albinw\AppData\Roaming\s3cmd.ini' DEBUG: ConfigParser: access_key->AK...17_chars...A DEBUG: ConfigParser: access_token-> DEBUG: ConfigParser: add_encoding_exts-> DEBUG: ConfigParser: add_headers-> DEBUG: ConfigParser: bucket_location->us-east-1 DEBUG: ConfigParser: ca_certs_file-> DEBUG: ConfigParser: cache_file-> DEBUG: ConfigParser: check_ssl_certificate->True DEBUG: ConfigParser: check_ssl_hostname->True DEBUG: ConfigParser: cloudfront_host->cloudfront.amazonaws.com DEBUG: ConfigParser: content_disposition-> DEBUG: ConfigParser: content_type-> DEBUG: ConfigParser: default_mime_type->binary/octet-stream DEBUG: ConfigParser: delay_updates->False DEBUG: ConfigParser: delete_after->False DEBUG: ConfigParser: delete_after_fetch->False DEBUG: ConfigParser: delete_removed->False DEBUG: ConfigParser: dry_run->False DEBUG: ConfigParser: enable_multipart->True DEBUG: ConfigParser: encoding->UTF-8 DEBUG: ConfigParser: encrypt->False DEBUG: ConfigParser: expiry_date-> DEBUG: ConfigParser: expiry_days-> DEBUG: ConfigParser: expiry_prefix-> DEBUG: ConfigParser: follow_symlinks->False DEBUG: ConfigParser: force->False DEBUG: ConfigParser: get_continue->False DEBUG: ConfigParser: gpg_command->C:\Program Files (x86)\GnuPG\bin\gpg.exe DEBUG: ConfigParser: gpg_decrypt->%(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s DEBUG: ConfigParser: gpg_encrypt->%(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s DEBUG: ConfigParser: gpg_passphrase->GA...7_chars...9 DEBUG: ConfigParser: guess_mime_type->True DEBUG: ConfigParser: host_base->s3.amazonaws.com DEBUG: ConfigParser: host_bucket->%(bucket)s.s3.amazonaws.com DEBUG: ConfigParser: human_readable_sizes->False DEBUG: ConfigParser: invalidate_default_index_on_cf->False DEBUG: ConfigParser: invalidate_default_index_root_on_cf->True DEBUG: ConfigParser: invalidate_on_cf->False DEBUG: ConfigParser: kms_key-> DEBUG: ConfigParser: limit->-1 DEBUG: ConfigParser: limitrate->0 DEBUG: ConfigParser: list_md5->False DEBUG: ConfigParser: log_target_prefix-> DEBUG: ConfigParser: long_listing->False DEBUG: ConfigParser: max_delete->-1 DEBUG: ConfigParser: mime_type-> DEBUG: ConfigParser: multipart_chunk_size_mb->15 DEBUG: ConfigParser: multipart_max_chunks->10000 DEBUG: ConfigParser: preserve_attrs->False DEBUG: ConfigParser: progress_meter->False DEBUG: ConfigParser: proxy_host-> DEBUG: ConfigParser: proxy_port->0 DEBUG: ConfigParser: put_continue->False DEBUG: ConfigParser: recursive->False DEBUG: ConfigParser: recv_chunk->65536 DEBUG: ConfigParser: reduced_redundancy->False DEBUG: ConfigParser: requester_pays->False DEBUG: ConfigParser: restore_days->1 DEBUG: ConfigParser: restore_priority->Standard DEBUG: ConfigParser: secret_key->CX...37_chars...q DEBUG: ConfigParser: send_chunk->65536 DEBUG: ConfigParser: server_side_encryption->False DEBUG: ConfigParser: signature_v2->False DEBUG: ConfigParser: signurl_use_https->False DEBUG: ConfigParser: simpledb_host->sdb.amazonaws.com DEBUG: ConfigParser: skip_existing->False DEBUG: ConfigParser: socket_timeout->300 DEBUG: ConfigParser: stats->False DEBUG: ConfigParser: stop_on_error->False DEBUG: ConfigParser: storage_class-> DEBUG: ConfigParser: throttle_max->100 DEBUG: ConfigParser: upload_id-> DEBUG: ConfigParser: urlencoding_mode->normal DEBUG: ConfigParser: use_http_expect->False DEBUG: ConfigParser: use_https->True DEBUG: ConfigParser: use_mime_magic->True DEBUG: ConfigParser: verbosity->WARNING DEBUG: ConfigParser: website_endpoint->http:// %(bucket)s.s3-website-%(location)s.amazonaws.com/ DEBUG: ConfigParser: website_error-> DEBUG: ConfigParser: website_index->index.html DEBUG: Updating Config.Config cache_file -> DEBUG: Updating Config.Config follow_symlinks -> False DEBUG: Updating Config.Config verbosity -> 10 DEBUG: Unicodising 'cp' using cp1252 DEBUG: Unicodising 's3://radius-crm-s3-us/xxxxxxxxxxxxxxxx-1469026373674/Exports/DST_InquiriesList_May/DST_Inquiries_List-May_2019.csv' using cp1252 DEBUG: Unicodising 'H://Documents/GAWG_R-Data_Folder/DST_Inquiries_List-May_2019.csv' using cp1252 DEBUG: Command: cp !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An unexpected error has occurred. Please try reproducing the error using the latest s3cmd code from the git master branch found at: https://github.com/s3tools/s3cmd and have a look at the known issues list: https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions If the error persists, please report the following lines (removing any private info as necessary) to: s3t...@li... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! DEBUG: Unicodising 's3cmd' using cp1252 DEBUG: Unicodising '-d' using cp1252 DEBUG: Unicodising 'cp' using cp1252 DEBUG: Unicodising 's3://radius-crm-s3-us/xxxxxxxxxxxxxxxx-1469026373674/Exports/DST_InquiriesList_May/DST_Inquiries_List-May_2019.csv' using cp1252 DEBUG: Unicodising 'H://Documents/GAWG_R-Data_Folder/DST_Inquiries_List-May_2019.csv' using cp1252 Invoked as: s3cmd -d cp s3://radius-crm-s3-us/xxxxxxxxxxxxxxxx-1469026373674/Exports/DST_InquiriesList_May/DST_Inquiries_List-May_2019.csv H://Documents/GAWG_R-Data_Folder/DST_Inquiries_List-May_2019.csv Problem: ValueError: H://Documents/GAWG_R-Data_Folder/DST_Inquiries_List-May_2019.csv: not a recognized URI S3cmd: 2.0.2 python: 2.7.16 (v2.7.16:413a49145e, Mar 4 2019, 01:37:19) [MSC v.1500 64 bit (AMD64)] Traceback (most recent call last): File "s3cmd", line 3172, in <module> report_exception(e) File "s3cmd", line 3047, in report_exception sys.stderr.write(u"environment LANG=%s\n" % unicodise_s(os.getenv("LANG"), 'ascii')) File "c:\Users\albinw\Documents\s3cmd-2.0.2\S3\Utils.py", line 321, in unicodise_s return unicodise(string, encoding, errors, True) File "c:\Users\albinw\Documents\s3cmd-2.0.2\S3\Utils.py", line 312, in unicodise return unicode(string, encoding, errors) TypeError: coercing to Unicode: need string or buffer, NoneType found c:\Users\albinw\Documents\s3cmd-2.0.2> -- *Todd Albin* Enrollment Analyst for Graduate and Adult Learning Enrollment University of Indianapolis al...@ui... 317.791.5929 <(317)%20788-5929> www.uindy.edu/graduate-admissions |