aws / aws/res

(2026.06 and earlier) TOCTOU symlink race condition in FileBrowser download route allows arbitrary file read as root

Open
#181 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
115
Forks
35
PR merge metrics
No merged PRs in 30d

Description

## Bug description
A race condition in the FileBrowser download route could allow unauthorized file access on the host. The fix eliminates the race window by atomically validating and opening the requested file in a single operation.
## Affected versions
* 2026.06 and earlier
## Mitigation

> **Tip:** We recommend running the following commands in [AWS CloudShell](https://console.aws.amazon.com/cloudshell/) in the same account and region where RES is deployed. CloudShell comes with pre-installed dependencies and helps eliminates other local environment conditions that may affect the patching process.

### For 2025.09:
1. Set up a Python 3.9 virtual environment and install all dependencies:

```
# Install prerequisites (CloudShell/Amazon Linux)
sudo yum install -y python3.9 patch

# Install prerequisites (Debian/Ubuntu)
# sudo apt install -y python3.9 python3.9-venv patch

# Create and activate virtual environment
python3.9 -m venv /tmp/patch-env
source /tmp/patch-env/bin/activate
pip install boto3 'setuptools<69.3'
```

2. Configure the AWS CLI for the account / region where RES is deployed, and make sure that you have S3 permissions to write to the bucket created by RES.
3. Download `patch.py` and `toctou-symlink-download.patch` by replacing `` with the directory path where to download both files and `` with the name of your RES environment in the command below:
1. The patch applies to RES 2025.09

```
OUTPUT_DIRECTORY=
ENVIRONMENT_NAME=
RES_VERSION=2025.09

mkdir -p ${OUTPUT_DIRECTORY}
curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch.py --output ${OUTPUT_DIRECTORY}/patch.py
curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/toctou-symlink-download.patch --output ${OUTPUT_DIRECTORY}/toctou-symlink-download.patch
```
4. Navigate to the directory where the patch script and patch file are downloaded. Run the following patch command:

```
python3 ${OUTPUT_DIRECTORY}/patch.py --environment-name ${ENVIRONMENT_NAME} --res-version ${RES_VERSION} --module cluster-manager --patch ${OUTPUT_DIRECTORY}/toctou-symlink-download.patch
```
5. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.

```
INSTANCE_ID=$(aws ec2 describe-instances \
--filters \
Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
--query "Reservations[0].Instances[0].InstanceId" \
--output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}
```
6. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name cluster-manager-asg. Wait until the new instance is launched successfully.
### For 2025.12, 2025.12.01, 2026.03 and 2026.06:
1. Set up a Python 3.13 virtual environment and install all dependencies:

```
# Install prerequisites (CloudShell/Amazon Linux)
sudo yum install -y python3.13

# Install prerequisites (Debian/Ubuntu)
# sudo apt install -y python3.13 python3.13-venv

# Create and activate virtual environment
python3.13 -m venv /tmp/patch-env
source /tmp/patch-env/bin/activate
pip install boto3
```
2. Create an S3 bucket with ACLs disabled. If you already have an S3 bucket with ACLs disabled, you can skip this step. Otherwise, create one using either the console or the CLI:
1. Option A (AWS Console):
1. Follow the [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html) guide in the Amazon S3 User Guide. When configuring the bucket:
2. Enter a bucket name and select the region where RES is deployed
3. Under Object Ownership, keep the default setting ACLs disabled (Bucket owner enforced)
4. Configure the remaining settings as needed (block public access, versioning, etc.)
5. Click Create bucket
2. Option B (CLI):
1. Replace `` and `` with your values and run the command:

```
aws s3api create-bucket \
--bucket \
--region \
--create-bucket-configuration LocationConstraint= \
--object-ownership BucketOwnerEnforced
```
Note: If your region is us-east-1, omit the —create-bucket-configuration parameter.

3. Download `patch_host.py` and `idea-cluster-manager-{res-version}-.tar.gz` by replacing `` with the directory to download the patch script, `` with the name of your RES environment, `` with the name of a ACLs disabled S3 bucket under the account/region where RES is deployed, `` with `2025.12`, `2025.12.01`, `2026.03` or `2026.06` in the command below, and `` with `Classic` or `GovCloud`:
1. The patch applies to 2025.12, 2025.12.01, 2026.03, 2026.06
2. Configure the AWS CLI for the account / region where RES is deployed, and make sure that you have S3 permissions to write to the bucket provided through `` and have DynamoDB access to access package uri of `.cluster-settings` table.

```
OUTPUT_DIRECTORY=
ENVIRONMENT_NAME=
RES_VERSION=
BUCKET_NAME=
PARTITION=

mkdir -p ${OUTPUT_DIRECTORY}
HASH=$(case $RES_VERSION in "2026.06") echo "f1362f23";; "2026.03") echo "769fc2cc";; "2025.12.01") echo "929fdbde";; "2025.12") echo "d7995e5f";; esac)
curl https://research-engineering-studio-us-east-1.s3.us-east-1.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch_host.py --output ${OUTPUT_DIRECTORY}/patch_host.py
curl https://research-engineering-studio-us-east-1.s3.us-east-1.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz --output ${OUTPUT_DIRECTORY}/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz
```
4. Before running the patch, note down the current package URI for rollback purposes. Save this value you will need it if you want to rollback later.

```
aws dynamodb get-item \
--table-name ${ENVIRONMENT_NAME}.cluster-settings \
--key '{"key": {"S": "cluster-manager.app_package_uri"}}' \
--query 'Item.value.S' --output text
```
5. Run the following patch command:

```
python3 ${OUTPUT_DIRECTORY}/patch_host.py --environment-name ${ENVIRONMENT_NAME} --module cluster-manager --zip-file ${OUTPUT_DIRECTORY}/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz --s3-bucket ${BUCKET_NAME} --partition ${PARTITION}
```
6. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.

```
INSTANCE_ID=$(aws ec2 describe-instances \
--filters \
Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
--query "Reservations[0].Instances[0].InstanceId" \
--output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}
```
7. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name cluster-manager-asg. Wait until the new instance is launched successfully.
### Rollback
**For 2025.09:** The patch script works as a toggle. Running the same command again will revert the patch. To rollback, re-run the exact command from the patching step:

1. If you're running this in a new terminal session, make sure to set the variables again:

```
OUTPUT_DIRECTORY=
ENVIRONMENT_NAME=
RES_VERSION=2025.09

python3 ${OUTPUT_DIRECTORY}/patch.py --environment-name ${ENVIRONMENT_NAME} --res-version ${RES_VERSION} --module cluster-manager --patch ${OUTPUT_DIRECTORY}/toctou-symlink-download.patch
```
2. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.

```
INSTANCE_ID=$(aws ec2 describe-instances \
--filters \
Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
--query "Reservations[0].Instances[0].InstanceId" \
--output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}
```
3. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name cluster-manager-asg. Wait until the new instance is launched successfully.

**For 2025.12 and later:** The patch modified the DynamoDB `*.app_package_uri` record to point to the patched tar.gz. To rollback:
1. Find the original package URI in the `{environment-name}.cluster-settings` DynamoDB table, if you followed the instructions, you should have saved the value. Otherwise, check CloudTrail or DDB item history for the previous value of the `cluster-manager.app_package_uri` key.
2. Restore the original value:

```
aws dynamodb update-item \
--table-name ${ENVIRONMENT_NAME}.cluster-settings \
--key '{"key": {"S": "cluster-manager.app_package_uri"}}' \
--update-expression "SET #val = :val" \
--expression-attribute-names '{"#val": "value"}' \
--expression-attribute-values '{":val": {"S": ""}}'
```
3. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.

```
INSTANCE_ID=$(aws ec2 describe-instances \
--filters \
Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
--query "Reservations[0].Instances[0].InstanceId" \
--output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}
```
4. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name cluster-manager-asg. Wait until the new instance is launched successfully.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the FileBrowser download route in the cluster-manager package and compare it with the referenced toctou-symlink-download.patch and patched cluster-manager tarballs. The issue names no source file or test, so trace the route and establish a regression test for the symlink race before changing implementation. Done means the download cannot read an unauthorized file through the race and the affected-version patch paths remain verifiable.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
backend, cloud, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.