blaze / blaze/odo

S3 problem: profiles for assuming AWS roles do not appear to work (boto3 problem?)

Open
#578 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1k
Forks
131
PR merge metrics
No merged PRs in 30d

Description

I need to get data from a S3 bucket that requires me to assume a role.

I have configured my setup as [per aws instructions](http://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html) and have a profile called "dev". I then `export AWS_PROFILE=dev` and this works fine with the aws cli, asking for an MFA when required and caching the results in the .aws/cli/cache folder.

I assumed I could do this in my python program (in the same environment)
`odo('s3://some-bucket/data.csv', 'sqlite:database.db::data')`

And it would also use the profile. But it seems not to find it, as I get a
`boto.provider.ProfileNotFoundError: Profile "dev" not found!` on line 296 of boto/provider.py.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the odo('s3://some-bucket/data.csv', 'sqlite:database.db::data') call with AWS_PROFILE=dev and compare it with the working AWS CLI setup. Start at boto/provider.py line 296 and trace how the profile is loaded, including the MFA cache; done means the configured profile is found and the S3-to-SQLite transfer no longer raises ProfileNotFoundError.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.