[C++] Writing to AWS S3 fails when using `aws sso` (13.0.0)
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
My org recently switched to using aws sso to configure all of our access and when making the swap, we noticed that we are no longer able to write to S3 when using these types of credentials. When explicitly setting the `AWS_SECRET_ACCESS_KEY` and `AWS_ACCESS_KEY_ID` with previous IAM User accounts, the problem does not exist. For reference, once authenticating with `aws sso login` and attempting to write some data, I am thrown the following error:
```
File "pyarrow/_dataset.pyx", line 3844, in pyarrow._dataset._filesystemdataset_write
File "pyarrow/error.pxi", line 115, in pyarrow.lib.check_status
OSError: When testing for existence of bucket 'my-s3-bucket': AWS Error ACCESS_DENIED during HeadBucket operation: No response body.
```
Note also that
`eval $(aws configure export-credentials --format env)` also does not seem to work as those are `AWS_SECRET_ACCESS_KEY` and `AWS_ACCESS_KEY_ID` from the sso login, which once exported also raise the same issue
I have confirmed that this is not an issue in 12.0.1, only 13.0.0
### Component(s)
Parquet
Contributor guide
Assessment
This issue has not been assessed yet.