aws / aws/aws-sdk-cpp

AWS SDK can't handle unusual object keys

Aperta
#1,924 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug p2
Lingua principale
C++
Stelle
2.2k
Fork
1.2k
Merge medio
4g 11h
PR unite (30g)
13

Descrizione

### Describe the bug

There are multiple issues with SDK that results in it's inability to handle unusual object keys (even if they are supported by underlying platform, be that AWS S3 or anything else). Examples of object keys:
```
a//
a///b
a/./b
a/..
and etc
```

1. Issue with repeating slashes is caused by [this line](https://github.com/aws/aws-sdk-cpp/blob/2770543b90a30412d0aca846b2c54847a48018b9/aws-cpp-sdk-core/source/http/URI.cpp#L154) -- `StringUtils::Split()` discards empty elements. This in turn causes us to access completely different object (e.g. `a/b` instead of `a///b`) with a whole shebang of possible results -- from race conditions to data loss.
2. Apparently, most systems treat _path_ component of URI in a special way -- before it gets processed path gets split into components (using `/` for separator) and if said component is `.` or `..` an attempt is made to get rid of it. After this URI gets used in downstream logic (which checks signature, etc). Naturally, this causes `Signature Mismatch` errors on server side because it observes slightly different URI than one sent by client. All because this SDK decided **not** to percent-encode `.` when building URI. Encoding it manually won't help because then it will be encoded second time by SDK. Basically, if your application tries to access an object with key `a/./b` -- it won't be able to.

Such object keys are relatively rare, but they happen. Not every application out there is written using AWS SDK. Some of them implemented this stuff properly and create objects this SDK can't deal with. This submission is result of couple days of investigations on "why my ECS -> S3 migration fails?" topic at major organization.

Bug is related somewhat to #1560 (with bugs in ECS implementation of V4 signature calcs it is even more convoluted)...

### Expected Behavior

I expect SDK to work regardless how weird/unusual symbols that comprise object key are.

### Current Behavior

It doesn't work. My app ends up hitting wrong objects, failing with `no such object`, HTTP codes 444, 404 and even 500 depending on storage device it is used against.

### Reproduction Steps

Should be pretty obvious from description.

### Possible Solution

1. Do not discard empty components in `URI::URLEncodePath` and other places
2. Encode `.` when building URI that will be used in HTTP request

### Additional Information/Context

_No response_

### AWS CPP SDK version used

1.7.116

### Compiler and Version used

VS2019/GCC

### Operating System and version

Win10, Linux

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia dalla riga collegata in aws-cpp-sdk-core/source/http/URI.cpp e segui URI::URLEncodePath e StringUtils::Split. Verifica come vengono gestiti gli slash ripetuti e i componenti punto prima di firmare e inviare le richieste. Il lavoro è completato quando chiavi di oggetti come a///b e a/./b raggiungono il servizio sottostante senza modifiche e non producono più mismatch o accessi all'oggetto errato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
cpp
Ambito
networking
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
38/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.