aws / aws/res

(2026.06) Snapshot apply fails when SSO is enabled on the source environment

Open
#177 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
115
Forks
35
PR merge metrics
No merged PRs in 30d

Description

## Bug description

When applying a snapshot to a RES 2026.06 environment, the apply process fails if SSO was enabled on the source environment. The snapshot contains the `sso_enabled=True` flag but does not include the required OIDC/SAML configuration parameters (e.g., `oidc_client_id`), causing the SSO auto-configuration step to fail with an error such as `SSO auto-configuration failed: [GENERAL_ERROR] oidc_client_id is required`.

## Affected versions

2026.06

## Mitigation

1. Create an S3 bucket with ACLs disabled.
2. Download `patch_host.py` and `idea-cluster-manager-2026.06-ed153413.tar.gz` by replacing `` with the directory to download the patch script, `` with the name of your RES environment, and `` with the name of an ACLs disabled S3 bucket under the account/region where RES is deployed, and `` with Classic or GovCloud:
1. The patch applies to RES 2026.06
2. The patch script requires [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html), Python 3.9.16 or above, and [Boto3](https://aws.amazon.com/sdk-for-python/).
3. Configure the AWS CLI for the account / region where RES is deployed, and make sure that you have S3 permissions to write to the bucket provided through ``.
```
OUTPUT_DIRECTORY=
ENVIRONMENT_NAME=
BUCKET_NAME=
PARTITION=

mkdir -p ${OUTPUT_DIRECTORY}
curl https://research-engineering-studio-us-east-1.s3.us-east-1.amazonaws.com/releases/2026.06/patch_scripts/patch_host.py --output ${OUTPUT_DIRECTORY}/patch_host.py
curl https://research-engineering-studio-us-east-1.s3.us-east-1.amazonaws.com/releases/2026.06/patch_scripts/patches/idea-cluster-manager-2026.06-ed153413.tar.gz --output ${OUTPUT_DIRECTORY}/idea-cluster-manager-2026.06-ed153413.tar.gz
```
3. Run the following patch command:
```
python3 ${OUTPUT_DIRECTORY}/patch_host.py --environment-name ${ENVIRONMENT_NAME} --module cluster-manager --zip-file ${OUTPUT_DIRECTORY}/idea-cluster-manager-2026.06-ed153413.tar.gz --s3-bucket ${BUCKET_NAME} --partition ${PARTITION}
```
4. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.
```
INSTANCE_ID=$(aws ec2 describe-instances \
--filters \
Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
Name=instance-state-name,Values=running \
--query "Reservations[0].Instances[0].InstanceId" \
--output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}
```
5. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name `-cluster-manager-asg`. Wait until the new instance is launched successfully.
2. Re-apply the snapshot. The patched cluster-manager will gracefully skip the SSO auto-configuration step and log a warning. After the snapshot is applied successfully, configure SSO manually through the RES web portal.

## Rollback

1. Restore the original package URI in DynamoDB (replace `` with the region where RES is deployed):
```
REGION=

aws dynamodb update-item \
--table-name ${ENVIRONMENT_NAME}.cluster-settings \
--key '{"key": {"S": "cluster-manager.app_package_uri"}}' \
--update-expression "SET #val = :val" \
--expression-attribute-names '{"#val": "value"}' \
--expression-attribute-values '{":val": {"S": "s3://research-engineering-studio-'${REGION}'/releases/2026.06/idea-cluster-manager-2026.06.tar.gz"}}'
```
2. Terminate the Cluster Manager instance to pick up the restored URI:
```
INSTANCE_ID=$(aws ec2 describe-instances \
--filters \
Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
Name=instance-state-name,Values=running \
--query "Reservations[0].Instances[0].InstanceId" \
--output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}
```

Contributor guide

Open the contributing guide

Research direction

Start with the cluster-manager snapshot-apply path and the SSO auto-configuration step described in the report; the payload names no repository source file or test. Reproduce with a snapshot from an SSO-enabled source, verify missing OIDC/SAML parameters are handled without failing, and confirm re-apply succeeds while logging a warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.