From: Rich M. <rm...@se...> - 2013-07-04 20:49:55
|
Does anyone have hints on using s3cmd with IAM roles? I have a role established and assigned to my EC2 instance, but after installing s3cmd I still get access denied. I don't see anything in the documentation. For example, do I need to create a special config file? Is there a command line parameter? Thanks |
From: Jason Q. <ja...@tc...> - 2013-07-04 21:54:34
|
Rich, I have noticed the same problem. In my case, while s3cmd would usually complain about not having permission when testing the credentials, it would work correctly when performing an action that the IAM credentials were permitted to. So, even though it says it wont work, it does. I've never looked at the code, but I gather they're listing all the buckets on the account or something to test the credentials, which if you've clamped down the IAM role/user/group to only access specific buckets, obviously wont work. Hope that helps, at least until one of the s3cmd guys gets to you... --Jason On 7/4/2013 4:19 PM, Rich Mogull wrote: > Does anyone have hints on using s3cmd with IAM roles? I have a role established and assigned to my EC2 instance, but after installing s3cmd I still get access denied. I don't see anything in the documentation. For example, do I need to create a special config file? Is there a command line parameter? > > Thanks > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > S3tools-general mailing list > S3t...@li... > https://lists.sourceforge.net/lists/listinfo/s3tools-general |
From: Sajan P. <sa...@no...> - 2013-07-04 22:07:23
Attachments:
smime.p7s
|
Here's something that should get your started. It would've helped if you showed us what your config currently looks like. { "Statement": [ { "Effect": "Allow", "Action": "*", "Resource": [ "arn:aws:s3:::your-bucket-name", "arn:aws:s3:::your-bucket-name/*" ], "Condition": {} } ] } Sajan Parikh /Owner, Noppix LLC/ e: sa...@no... p: (563) 726-0371 Noppix LLC Logo On 07/04/2013 03:19 PM, Rich Mogull wrote: > Does anyone have hints on using s3cmd with IAM roles? I have a role established and assigned to my EC2 instance, but after installing s3cmd I still get access denied. I don't see anything in the documentation. For example, do I need to create a special config file? Is there a command line parameter? > > Thanks > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > S3tools-general mailing list > S3t...@li... > https://lists.sourceforge.net/lists/listinfo/s3tools-general |
From: Rich M. <rm...@se...> - 2013-07-06 02:45:45
|
Sajan, Here;s the policy I’m using that doesn’t seem to work. This is *before* running —config, since I’m trying to figure out how to script a cloud-init download of some security credentials. Running "s3cmd ls” gives me the access denied error. Thank you for the help, { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:Get*", "s3:List*" ], "Resource": "arn:aws:s3:::<my bucket>" } ] } Rich Mogull rm...@se... AIM: Securosis Skype: rmogull work+blog: http://securosis.com On Jul 4, 2013, at 1:58 PM, Sajan Parikh <sa...@no...> wrote: > Here's something that should get your started. It would've helped if you showed us what your config currently looks like. > > { > "Statement": [ > { > "Effect": "Allow", > "Action": "*", > "Resource": [ > "arn:aws:s3:::your-bucket-name", > "arn:aws:s3:::your-bucket-name/*" > ], > "Condition": {} > } > ] > } > Sajan Parikh > Owner, Noppix LLC > > e: sa...@no... > p: (563) 726-0371 > > <emailsiglogo.png> > On 07/04/2013 03:19 PM, Rich Mogull wrote: >> Does anyone have hints on using s3cmd with IAM roles? I have a role established and assigned to my EC2 instance, but after installing s3cmd I still get access denied. I don't see anything in the documentation. For example, do I need to create a special config file? Is there a command line parameter? >> >> Thanks >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> S3tools-general mailing list >> S3t...@li... >> https://lists.sourceforge.net/lists/listinfo/s3tools-general > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev_______________________________________________ > S3tools-general mailing list > S3t...@li... > https://lists.sourceforge.net/lists/listinfo/s3tools-general |
From: Sajan P. <sa...@no...> - 2013-07-07 21:58:39
|
Did you try the config I posted to the list a while ago? I'd been using that config for a long while without any issues, even before any sort of support in S3Tools. I created an IAM user, attached the policy I posted before and used the key and secret key for that particular user like normal in s3cmd --configure. Has worked like a charm for a while, and I haven't updated s3cmd in months. Sajan Parikh /Owner, Noppix LLC/ e: sa...@no... p: (563) 726-0371 Noppix LLC Logo On 07/05/2013 09:45 PM, Rich Mogull wrote: > Sajan, > > Here;s the policy I'm using that doesn't seem to work. This is > *before* running ---config, since I'm trying to figure out how to > script a cloud-init download of some security credentials. Running > "s3cmd ls" gives me the access denied error. > > Thank you for the help, > > { > "Version": "2012-10-17", > "Statement": [ > { > "Effect": "Allow", > "Action": [ > "s3:Get*", > "s3:List*" > ], > "Resource": "arn:aws:s3:::<my bucket>" > } > ] > } > > Rich Mogull > rm...@se... <mailto:rm...@se...> > AIM: Securosis > Skype: rmogull > work+blog: http://securosis.com > > On Jul 4, 2013, at 1:58 PM, Sajan Parikh <sa...@no... > <mailto:sa...@no...>> wrote: > >> Here's something that should get your started. It would've helped if >> you showed us what your config currently looks like. >> >> { >> "Statement": [ >> { >> "Effect": "Allow", >> "Action": "*", >> "Resource": [ >> "arn:aws:s3:::your-bucket-name", >> "arn:aws:s3:::your-bucket-name/*" >> ], >> "Condition": {} >> } >> ] >> } >> Sajan Parikh >> /Owner, Noppix LLC/ >> >> e:sa...@no... >> p: (563) 726-0371 >> >> <emailsiglogo.png> >> On 07/04/2013 03:19 PM, Rich Mogull wrote: >>> Does anyone have hints on using s3cmd with IAM roles? I have a role established and assigned to my EC2 instance, but after installing s3cmd I still get access denied. I don't see anything in the documentation. For example, do I need to create a special config file? Is there a command line parameter? >>> >>> Thanks >>> >>> ------------------------------------------------------------------------------ >>> ThisSF.net <http://SF.net> email is sponsored by Windows: >>> >>> Build for Windows Store. >>> >>> http://p.sf.net/sfu/windows-dev2dev >>> _______________________________________________ >>> S3tools-general mailing list >>> S3t...@li... >>> https://lists.sourceforge.net/lists/listinfo/s3tools-general >> >> ------------------------------------------------------------------------------ >> ThisSF.net <http://sf.net/>email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev_______________________________________________ >> S3tools-general mailing list >> S3t...@li... >> <mailto:S3t...@li...> >> https://lists.sourceforge.net/lists/listinfo/s3tools-general > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > > > _______________________________________________ > S3tools-general mailing list > S3t...@li... > https://lists.sourceforge.net/lists/listinfo/s3tools-general |
From: Rich M. <rm...@se...> - 2013-07-08 16:30:45
|
Thanks Sajan, It isn’t the IAM configuration that is the problem. What I’m trying to do is use an AWS IAM role, which means I wouldn’t need to create a user account or embed static credentials into the s3cmd config file. With a role assigned to the EC2 instance, any tools that support roles are automatically provided the needed credentials when they run. The access and secret key are temporary, and not stored in the instance. This is really powerful for autoscaling and bootstrapping securely. The alternative (which someone posted) is to do some scripting to pull the temp credentials into s3cmd when needed, which is what I’ll try next unless anyone has suggestions for getting IAM role support working (in alpha 3). That’s similar to your user-based approach, but will use temporary credentials instead. Then I can revoke the role after the system is up and running and not worry about affecting anything else. Thanks, Rich Mogull rm...@se... AIM: Securosis Skype: rmogull work+blog: http://securosis.com On Jul 7, 2013, at 2:58 PM, Sajan Parikh <sa...@no...> wrote: > Did you try the config I posted to the list a while ago? I'd been using that config for a long while without any issues, even before any sort of support in S3Tools. > > I created an IAM user, attached the policy I posted before and used the key and secret key for that particular user like normal in s3cmd --configure. > > Has worked like a charm for a while, and I haven't updated s3cmd in months. > > Sajan Parikh > Owner, Noppix LLC > > e: sa...@no... > p: (563) 726-0371 > > <emailsiglogo.png> > On 07/05/2013 09:45 PM, Rich Mogull wrote: >> Sajan, >> >> Here;s the policy I’m using that doesn’t seem to work. This is *before* running —config, since I’m trying to figure out how to script a cloud-init download of some security credentials. Running "s3cmd ls” gives me the access denied error. >> >> Thank you for the help, >> >> { >> "Version": "2012-10-17", >> "Statement": [ >> { >> "Effect": "Allow", >> "Action": [ >> "s3:Get*", >> "s3:List*" >> ], >> "Resource": "arn:aws:s3:::<my bucket>" >> } >> ] >> } >> >> >> >> On Jul 4, 2013, at 1:58 PM, Sajan Parikh <sa...@no...> wrote: >> >>> Here's something that should get your started. It would've helped if you showed us what your config currently looks like. >>> >>> { >>> "Statement": [ >>> { >>> "Effect": "Allow", >>> "Action": "*", >>> "Resource": [ >>> "arn:aws:s3:::your-bucket-name", >>> "arn:aws:s3:::your-bucket-name/*" >>> ], >>> "Condition": {} >>> } >>> ] >>> } >>> Sajan Parikh >>> Owner, Noppix LLC >>> >>> e: sa...@no... >>> p: (563) 726-0371 >>> >>> <emailsiglogo.png> >>> On 07/04/2013 03:19 PM, Rich Mogull wrote: >>>> Does anyone have hints on using s3cmd with IAM roles? I have a role established and assigned to my EC2 instance, but after installing s3cmd I still get access denied. I don't see anything in the documentation. For example, do I need to create a special config file? Is there a command line parameter? >>>> >>>> Thanks >>>> >>>> ------------------------------------------------------------------------------ >>>> This SF.net email is sponsored by Windows: >>>> >>>> Build for Windows Store. >>>> >>>> http://p.sf.net/sfu/windows-dev2dev >>>> _______________________________________________ >>>> S3tools-general mailing list >>>> S3t...@li... >>>> https://lists.sourceforge.net/lists/listinfo/s3tools-general >>> >>> ------------------------------------------------------------------------------ >>> This SF.net email is sponsored by Windows: >>> >>> Build for Windows Store. >>> >>> http://p.sf.net/sfu/windows-dev2dev_______________________________________________ >>> S3tools-general mailing list >>> S3t...@li... >>> https://lists.sourceforge.net/lists/listinfo/s3tools-general >> >> >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> >> >> _______________________________________________ >> S3tools-general mailing list >> S3t...@li... >> https://lists.sourceforge.net/lists/listinfo/s3tools-general > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev_______________________________________________ > S3tools-general mailing list > S3t...@li... > https://lists.sourceforge.net/lists/listinfo/s3tools-general |
From: Rich M. <rm...@se...> - 2013-07-08 21:00:58
|
Okay- thanks for the help everyone. For some reason it suddenly started working properly today. I assume I made some sort of an error someplace, but it behaved as expected suddenly, in an instance throwing errors on the same command minutes earlier. I suspect a policy sync/application issue of some sort. My bad, On Jul 8, 2013, at 9:30 AM, Rich Mogull <rm...@se...> wrote: > Thanks Sajan, > > It isn’t the IAM configuration that is the problem. What I’m trying to do is use an AWS IAM role, which means I wouldn’t need to create a user account or embed static credentials into the s3cmd config file. > > With a role assigned to the EC2 instance, any tools that support roles are automatically provided the needed credentials when they run. The access and secret key are temporary, and not stored in the instance. This is really powerful for autoscaling and bootstrapping securely. > > The alternative (which someone posted) is to do some scripting to pull the temp credentials into s3cmd when needed, which is what I’ll try next unless anyone has suggestions for getting IAM role support working (in alpha 3). That’s similar to your user-based approach, but will use temporary credentials instead. Then I can revoke the role after the system is up and running and not worry about affecting anything else. > > Thanks, > > > > On Jul 7, 2013, at 2:58 PM, Sajan Parikh <sa...@no...> wrote: > >> Did you try the config I posted to the list a while ago? I'd been using that config for a long while without any issues, even before any sort of support in S3Tools. >> >> I created an IAM user, attached the policy I posted before and used the key and secret key for that particular user like normal in s3cmd --configure. >> >> Has worked like a charm for a while, and I haven't updated s3cmd in months. >> >> Sajan Parikh >> Owner, Noppix LLC >> >> e: sa...@no... >> p: (563) 726-0371 >> >> <emailsiglogo.png> >> On 07/05/2013 09:45 PM, Rich Mogull wrote: >>> Sajan, >>> >>> Here;s the policy I’m using that doesn’t seem to work. This is *before* running —config, since I’m trying to figure out how to script a cloud-init download of some security credentials. Running "s3cmd ls” gives me the access denied error. >>> >>> Thank you for the help, >>> >>> { >>> "Version": "2012-10-17", >>> "Statement": [ >>> { >>> "Effect": "Allow", >>> "Action": [ >>> "s3:Get*", >>> "s3:List*" >>> ], >>> "Resource": "arn:aws:s3:::<my bucket>" >>> } >>> ] >>> } >>> >>> >>> >>> On Jul 4, 2013, at 1:58 PM, Sajan Parikh <sa...@no...> wrote: >>> >>>> Here's something that should get your started. It would've helped if you showed us what your config currently looks like. >>>> >>>> { >>>> "Statement": [ >>>> { >>>> "Effect": "Allow", >>>> "Action": "*", >>>> "Resource": [ >>>> "arn:aws:s3:::your-bucket-name", >>>> "arn:aws:s3:::your-bucket-name/*" >>>> ], >>>> "Condition": {} >>>> } >>>> ] >>>> } >>>> Sajan Parikh >>>> Owner, Noppix LLC >>>> >>>> e: sa...@no... >>>> p: (563) 726-0371 >>>> >>>> <emailsiglogo.png> >>>> On 07/04/2013 03:19 PM, Rich Mogull wrote: >>>>> Does anyone have hints on using s3cmd with IAM roles? I have a role established and assigned to my EC2 instance, but after installing s3cmd I still get access denied. I don't see anything in the documentation. For example, do I need to create a special config file? Is there a command line parameter? >>>>> >>>>> Thanks >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> This SF.net email is sponsored by Windows: >>>>> >>>>> Build for Windows Store. >>>>> >>>>> http://p.sf.net/sfu/windows-dev2dev >>>>> _______________________________________________ >>>>> S3tools-general mailing list >>>>> S3t...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/s3tools-general >>>> >>>> ------------------------------------------------------------------------------ >>>> This SF.net email is sponsored by Windows: >>>> >>>> Build for Windows Store. >>>> >>>> http://p.sf.net/sfu/windows-dev2dev_______________________________________________ >>>> S3tools-general mailing list >>>> S3t...@li... >>>> https://lists.sourceforge.net/lists/listinfo/s3tools-general >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> This SF.net email is sponsored by Windows: >>> >>> Build for Windows Store. >>> >>> http://p.sf.net/sfu/windows-dev2dev >>> >>> >>> _______________________________________________ >>> S3tools-general mailing list >>> S3t...@li... >>> https://lists.sourceforge.net/lists/listinfo/s3tools-general >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev_______________________________________________ >> S3tools-general mailing list >> S3t...@li... >> https://lists.sourceforge.net/lists/listinfo/s3tools-general > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev_______________________________________________ > S3tools-general mailing list > S3t...@li... > https://lists.sourceforge.net/lists/listinfo/s3tools-general |