aws / aws/aws-sdk-cpp

AWS SDK can't handle unusual object keys

Open
#1,924 2 comments 0 reactions 0 assignees View on GitHub
bug p2
Dominant language
C++
Stars
2.2k
Forks
1.2k
Avg merge
3d 14h
Merged PRs (30d)
12

Description

### Describe the bug

There are multiple issues with SDK that results in it's inability to handle unusual object keys (even if they are supported by underlying platform, be that AWS S3 or anything else). Examples of object keys:
```
a//
a///b
a/./b
a/..
and etc
```

1. Issue with repeating slashes is caused by [this line](https://github.com/aws/aws-sdk-cpp/blob/2770543b90a30412d0aca846b2c54847a48018b9/aws-cpp-sdk-core/source/http/URI.cpp#L154) -- `StringUtils::Split()` discards empty elements. This in turn causes us to access completely different object (e.g. `a/b` instead of `a///b`) with a whole shebang of possible results -- from race conditions to data loss.
2. Apparently, most systems treat _path_ component of URI in a special way -- before it gets processed path gets split into components (using `/` for separator) and if said component is `.` or `..` an attempt is made to get rid of it. After this URI gets used in downstream logic (which checks signature, etc). Naturally, this causes `Signature Mismatch` errors on server side because it observes slightly different URI than one sent by client. All because this SDK decided **not** to percent-encode `.` when building URI. Encoding it manually won't help because then it will be encoded second time by SDK. Basically, if your application tries to access an object with key `a/./b` -- it won't be able to.

Such object keys are relatively rare, but they happen. Not every application out there is written using AWS SDK. Some of them implemented this stuff properly and create objects this SDK can't deal with. This submission is result of couple days of investigations on "why my ECS -> S3 migration fails?" topic at major organization.

Bug is related somewhat to #1560 (with bugs in ECS implementation of V4 signature calcs it is even more convoluted)...

### Expected Behavior

I expect SDK to work regardless how weird/unusual symbols that comprise object key are.

### Current Behavior

It doesn't work. My app ends up hitting wrong objects, failing with `no such object`, HTTP codes 444, 404 and even 500 depending on storage device it is used against.

### Reproduction Steps

Should be pretty obvious from description.

### Possible Solution

1. Do not discard empty components in `URI::URLEncodePath` and other places
2. Encode `.` when building URI that will be used in HTTP request

### Additional Information/Context

_No response_

### AWS CPP SDK version used

1.7.116

### Compiler and Version used

VS2019/GCC

### Operating System and version

Win10, Linux

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.