Cannot use FIPS s3 endpoints while using SSO.
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 4.6k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 13
Description
### Describe the bug
The ultimate issue I'm raising is that I am unable to use FIPS endpoints for s3 while also utilizing sso as my authentication method.
### Expected Behavior
I'd expect one of the methods that I tried to allow SSO authentication while seamlessly being able to utilize s3's FIPS endpoints.
### Current Behavior
Setting the `AWS_USE_FIPS_ENDPOINT` env variable to true, causes authentication attempts to use an SSO URL that does not exist. Same thing occurs if you set the `use_fips_endpoint` option in the aws `config` file.
```
$ export AWS_USE_FIPS_ENDPOINT=true
$ aws sso login --profile
Could not connect to the endpoint URL: "https://oidc-fips..amazonaws.com/device_authorization"
```
This happens because there is [no FIPS endpoint for SSO](https://aws.amazon.com/compliance/fips/#:~:text=AWS%20IAM%20Identity%20Center%20(Successor%20to%20AWS%20Single%20Sign%2DOn)) (in fact, the sso FIPS endpoints for govcloud do not use any FIPS identifiers in their FQDNs), you're intended to use the non-FIPS endpoint for SSO and then utilize the FIPS endpoint for the specific service you are attempting to access. This previous statement would be fine if I was able to provide service specific settings for FIPS, but that also does not work. I.e. using this setting in the aws `config` file.
```
s3 =
use_fips_endpoint = true
```
This setting is completely ignored when using s3 from the cli.
Attempts to utilize the endpoint override option `--endpoint-url` is unsuccessful, because [s3 FIPS endpoints require the Virtual Host-Style addressing](https://aws.amazon.com/compliance/fips/#:~:text=Virtual%20Hosted%2DStyle%20addressing.)
The s3 CLI expects a **non**-virtual host-style address in the `--endpoint-url` and is expecting the bucket name to be passed as a separate parameter:
```
aws s3 ls --profile --endpoint-url https://s3-fips..amazonaws.com
```
However, FIPS doesn't support `https://s3-fips..amazonaws.com`, which causes the above command to fail.
```
$ aws s3 ls --profile --endpoint-url https://s3-fips..amazonaws.com
Could not connect to the endpoint URL: "https://s3-fips..amazonaws.com/?list-type=2&prefix=&delimiter=%2F&encoding-type=url"
```
Because FIPS requires Virtual Host-Style addressing, it would only support this type of command line:
```
aws s3 ls --profile --endpoint-url https://.s3-fips.us-east-1.amazonaws.com
```
Which does authenticate successfully, but the CLI program is expecting it to return a `Bucket` property (assuming because its expecting the `--endpoint-url` parameter to point to a URL that would have provided a listing of buckets, but the only supported FIPS endpoint is the bucket specific address)
```
aws s3 ls --profile --endpoint-url https://.s3-fips..amazonaws.com
'Buckets'
```
With the `--debug` option on, you can see that the `'Buckets'` string being returned is actually an error for the key `Buckets`.
```
2024-05-13 10:08:54,289 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.ListBuckets: calling handler >
2024-05-13 10:08:54,289 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2024-05-13 10:08:54,289 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.ListBuckets: calling handler >
2024-05-13 10:08:54,289 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.ListBuckets: calling handler
2024-05-13 10:08:54,289 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.ListBuckets: calling handler >
2024-05-13 10:08:54,290 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
File "awscli/clidriver.py", line 460, in main
File "awscli/customizations/commands.py", line 151, in __call__
File "awscli/customizations/commands.py", line 205, in __call__
File "awscli/customizations/s3/subcommands.py", line 528, in _run_main
File "awscli/customizations/s3/subcommands.py", line 594, in _list_all_buckets
KeyError: 'Buckets'
'Buckets'
```
The final option is to manually set the FIPS settings `AWS_USE_FIPS_ENDPOINT` (remove the setting to SSO and then add the setting back when using s3 commands, however, during a substantially long file transfer, SSO token will need to be refreshed, and will attempt to utilize FIPS endpoint for the refresh and fails.
### Reproduction Steps
### Failing SSO auth
- replace with a valid profile in your `config` file.
```
$ export AWS_USE_FIPS_ENDPOINT=true
$ aws sso login --profile
```
### Ignoring s3 `use_fips_endpoint` option
1. Set your `config` file with the following settings.
```
[profile Test]
.
.
.
s3 =
use_fips_endpoint = true
```
2. Run the following commands (ensure --debug is added to see that it does not utilize a fips endpoint)
```
aws sso login --profile Test
aws s3 ls --profile Test --debug
```
### Specifying `--endpoint-url` parameter
This will fail because s3 fips only supports Virtual host-address in the URL.
```
aws s3 ls --profile --endpoint-url https://s3-fips..amazonaws.com
```
Specifying the virtual host-name address (as required by FIPS) will fail with `Buckets` output. (utilize `--debug` to see the stack trace.)
```
aws s3 ls --profile --endpoint-url https://.s3-fips.us-east-1.amazonaws.com --debug
```
### Possible Solution
Some possible solutions:
- Don't assume a FIPS endpoint for SSO when `AWS_USE_FIPS_ENDPOINT` or `use_fips_endpoint` global options are set.
- Allow SSO specific settings to override FIPS global setting.
- Allow S3 to utilize a FIPS specific option in the aws `config` file.
- Allow `--endpoint-url` to handle Virtual Host-Only addresses to accommodate the FIPS limitation.
### Additional Information/Context
_No response_
### CLI version used
aws-cli/2.15.37 Python/3.11.8 Linux/6.5.0-9021-oem exe/x86_64.ubuntu.22 prompt/off
### Environment details (OS name and version, etc.)
Ubuntu with Linux 6.5.0 x86_64
Contributor guide
Assessment
This issue has not been assessed yet.