#148 s3cmd - ERROR: Test failed: 403 (AccessDenied): Access Denied

Malfunction
closed-invalid
nobody
None
5
2015-02-18
2014-06-16
onekgenome
No

Hi,

I am not able to run s3cmd on my ec2 instance. I am trying to access a public s3 dataset (s3.amazonaws.com/1000genomes). The dataset is public and should be accessible by anyone.
What am I doing wrong?

Thanks

Here is the error :

ubuntu@ip-myIP:~$ s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3
Access Key: <removed>
Secret Key: <removed>

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP and can't be used if you're behind a proxy
Use HTTPS protocol [No]:

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't conect to S3 directly
HTTP Proxy server name:

New settings:
Access Key: <removed>
Secret Key: <removed>
Encryption password:
Path to GPG program: /usr/bin/gpg
Use HTTPS protocol: False
HTTP Proxy server name:
HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] Y
Please wait...
ERROR: Test failed: 403 (AccessDenied): Access Denied

Retry configuration? [Y/n] n

Save settings? [y/N] y
Configuration saved to '/home/ubuntu/.s3cfg'

=================

ubuntu@ip-myIP:~$ s3cmd ls s3://1000genomes/
ERROR: Access to bucket '1000genomes' was denied

I am not sure how to solve this?

Discussion

  • Matt Domsch

    Matt Domsch - 2014-06-17

    You must have set up AWS access key and secret key on AWS, and then put those values into the s3cmd configuration. The S3 API requires that you use AWS keys to perform any actions, even if the data available through those actions is public.

    After setting up my access key and secret key, here's what I see:

    $ s3cmd ls s3://1000genomes/
                           DIR   s3://1000genomes/alignment_indices/
                           DIR   s3://1000genomes/changelog_details/
                           DIR   s3://1000genomes/complete_genomics_indices/
                           DIR   s3://1000genomes/data/
                           DIR   s3://1000genomes/phase1/
                           DIR   s3://1000genomes/pilot_data/
                           DIR   s3://1000genomes/release/
                           DIR   s3://1000genomes/sequence_indices/
                           DIR   s3://1000genomes/technical/
    2014-01-30 19:14    216354   s3://1000genomes/CHANGELOG
    2013-08-06 16:11     15977   s3://1000genomes/README.alignment_data
    2014-01-30 11:13      5289   s3://1000genomes/README.analysis_history
    2014-01-31 03:44      5967   s3://1000genomes/README.complete_genomics_data
    2013-08-06 16:11       935   s3://1000genomes/README.ebi_aspera_info
    2013-08-06 16:11      8408   s3://1000genomes/README.ftp_structure
    2013-08-06 16:11      2082   s3://1000genomes/README.pilot_data
    2014-01-31 13:52      1937   s3://1000genomes/README.populations
    2013-08-06 16:11      7857   s3://1000genomes/README.sequence_data
    2013-08-06 16:11  10220497   s3://1000genomes/alignment.index
    2013-08-06 16:11  18497500   s3://1000genomes/analysis.sequence.index
    2013-08-06 16:11  48886866   s3://1000genomes/current.tree
    2013-08-06 16:11    397947   s3://1000genomes/exome.alignment.index
    2013-08-06 16:11  13248765   s3://1000genomes/sequence.index
    
     
  • Matt Domsch

    Matt Domsch - 2014-06-17
    • status: open --> closed-invalid
     
  • onekgenome

    onekgenome - 2014-06-17

    Hi Matt,

    Actually I did input access and secret keys, I just deleted them for this posting.
    Do I need to set permissions somewhere else? This is the .s3cfg file.

    1 [default]
    2 access_key = HIDDEN
    3 bucket_location = US
    4 cloudfront_host = cloudfront.amazonaws.com
    5 cloudfront_resource = /2010-07-15/distribution
    6 default_mime_type = binary/octet-stream
    7 delete_removed = False
    8 dry_run = False
    9 encoding = UTF-8
    10 encrypt = False
    11 follow_symlinks = False
    12 force = False
    13 get_continue = False
    14 gpg_command = /usr/bin/gpg
    15 gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
    16 gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
    17 gpg_passphrase =
    18 guess_mime_type = True
    19 host_base = s3.amazonaws.com
    20 host_bucket = %(bucket)s.s3.amazonaws.com
    21 human_readable_sizes = False
    22 list_md5 = False
    23 log_target_prefix =
    24 preserve_attrs = True
    25 progress_meter = True
    26 proxy_host =
    27 proxy_port = 0
    28 recursive = False
    29 recv_chunk = 4096
    30 reduced_redundancy = False
    31 secret_key = HIDDEN
    32 send_chunk = 4096
    33 simpledb_host = sdb.amazonaws.com
    34 skip_existing = False
    35 socket_timeout = 10
    36 urlencoding_mode = normal
    37 use_https = False
    38 verbosity = WARNING

    The status of this post has been set to closed. Did I post in wrong place?

    Best

     
  • onekgenome

    onekgenome - 2014-06-17

    Okay, figure this out. I had to attach policy under permissions in IAM.

    Thanks

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks