mapbox / mapbox/dynamodb-replicator
incremental-snapshot doesnt handle s3 timeouts well
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 131
- Forks
- 46
- PR merge metrics
- No merged PRs in 30d
Description
`incremental-snapshot.js` doesnt seem to handle s3 timeouts very well - leaving a broken (partial, missing, or otherwise) snapshot in it's wake:
```bin/incremental-snapshot.js s3://$BackupBucket/$BackupPrefix/$TABLE s3://$BackupBucket/${TABLE}-snapshot```
```create snapshot from bucket
[Tue, 10 Jan 2017 17:12:49 GMT] [info] [incremental-snapshot] Starting snapshot from s3://dsr-ddb-rep-testing/testprefix/showdownlive_gamedata_dev-01 to s3://dsr-ddb-rep-testing/showdownlive_gamedata_dev-01-snapshot
[Tue, 10 Jan 2017 17:12:59 GMT] [info] [incremental-snapshot] Starting upload of part #0, 0 bytes uploaded, 3000 items uploaded @ 297.65 items/s
[Tue, 10 Jan 2017 17:13:06 GMT] [error] [incremental-snapshot] TimeoutError: Connection timed out after 1000ms
at ClientRequest. (/Users/draistrick/git/github/dynamodb-replicator/node_modules/aws-sdk/lib/http/node.js:56:34)
at ClientRequest.g (events.js:286:16)
at emitNone (events.js:86:13)
at ClientRequest.emit (events.js:185:7)
at TLSSocket.emitTimeout (_http_client.js:614:10)
at TLSSocket.g (events.js:286:16)
at emitNone (events.js:91:20)
at TLSSocket.emit (events.js:185:7)
at TLSSocket.Socket._onTimeout (net.js:333:8)
at tryOnTimeout (timers.js:228:11)
message: Connection timed out after 1000ms
code: NetworkingError
region: us-west-2
hostname: dsr-ddb-rep-testing.s3-us-west-2.amazonaws.com
```
this case also exits 0, instead of with an error...so hard to handle externally
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with bin/incremental-snapshot.js and reproduce the S3 timeout using the command shown in the issue. Trace how the timeout is reported after part #0 begins uploading. Done means a timeout cannot leave a partial snapshot unnoticed and the command exits with an error instead of status 0.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, javascript
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100