awslabs / awslabs/aws-sdk-rust

Unexplained blocking behavior with DynamoDB client

Open
#611 14 comments 0 reactions 0 assignees View on GitHub
dependencies p3
Dominant language
Rust
Stars
3.3k
Forks
290
Avg merge
1d 12h
Merged PRs (30d)
3

Description

### Describe the bug

I've encountered a problem that I cannot explain, where a deadlock happens. See reproduction steps for more details.

An interesting note is that if I create a new DynamoDB client, instead of `clone`ing, I do not see the issue.

### Expected Behavior

I would have expected to see something like:
```
Hello, world!
Sync background scan size: 7528
Async background scan size: 7528
Sync background scan size: 7528
Sync background scan size: 7528
Sync background scan size: 7528
Sync background scan size: 7528
Sync background scan size: 7528
Sync background scan size: 7528
...
```

### Current Behavior

I'm seeing a deadlock:
```
Hello, world!
Sync background scan size: 7528
Async background scan size: 7528

^C
```

### Reproduction Steps

```
async fn async_background_scan(client: aws_sdk_dynamodb::Client) {
let scan = client.scan().table_name("Buckets").send().await.unwrap();
println!("Async background scan size: {}", scan.count());
loop {}
}

fn blocking_background_scan(client: aws_sdk_dynamodb::Client) {
loop {
let scan = futures::executor::block_on(client.scan().table_name("Buckets").send()).unwrap();
println!("Sync background scan size: {}", scan.count());
}
}

#[tokio::main]
async fn main() {
let config = aws_config::load_from_env().await;
let ddb_client = aws_sdk_dynamodb::Client::new(&config);

tokio::spawn(async_background_scan(ddb_client.clone()));
tokio::task::spawn_blocking(move || blocking_background_scan(ddb_client));

println!("Hello, world!");
loop {}
}
```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### Version

```text
├── aws-config v0.47.0
│ ├── aws-http v0.47.0
│ │ ├── aws-smithy-http v0.47.0
│ │ │ ├── aws-smithy-types v0.47.0
│ │ ├── aws-smithy-types v0.47.0 (*)
│ │ ├── aws-types v0.47.0
│ │ │ ├── aws-smithy-async v0.47.0
│ │ │ ├── aws-smithy-client v0.47.0
│ │ │ │ ├── aws-smithy-async v0.47.0 (*)
│ │ │ │ ├── aws-smithy-http v0.47.0 (*)
│ │ │ │ ├── aws-smithy-http-tower v0.47.0
│ │ │ │ │ ├── aws-smithy-http v0.47.0 (*)
│ │ │ │ ├── aws-smithy-types v0.47.0 (*)
│ │ │ ├── aws-smithy-http v0.47.0 (*)
│ │ │ ├── aws-smithy-types v0.47.0 (*)
│ ├── aws-sdk-sso v0.17.0
│ │ ├── aws-endpoint v0.47.0
│ │ │ ├── aws-smithy-http v0.47.0 (*)
│ │ │ ├── aws-types v0.47.0 (*)
│ │ ├── aws-http v0.47.0 (*)
│ │ ├── aws-sig-auth v0.47.0
│ │ │ ├── aws-sigv4 v0.47.0
│ │ │ │ ├── aws-smithy-http v0.47.0 (*)
│ │ │ ├── aws-smithy-http v0.47.0 (*)
│ │ │ ├── aws-types v0.47.0 (*)
│ │ ├── aws-smithy-async v0.47.0 (*)
│ │ ├── aws-smithy-client v0.47.0 (*)
│ │ ├── aws-smithy-http v0.47.0 (*)
│ │ ├── aws-smithy-http-tower v0.47.0 (*)
│ │ ├── aws-smithy-json v0.47.0
│ │ │ └── aws-smithy-types v0.47.0 (*)
│ │ ├── aws-smithy-types v0.47.0 (*)
│ │ ├── aws-types v0.47.0 (*)
│ ├── aws-sdk-sts v0.17.0
│ │ ├── aws-endpoint v0.47.0 (*)
│ │ ├── aws-http v0.47.0 (*)
│ │ ├── aws-sig-auth v0.47.0 (*)
│ │ ├── aws-smithy-async v0.47.0 (*)
│ │ ├── aws-smithy-client v0.47.0 (*)
│ │ ├── aws-smithy-http v0.47.0 (*)
│ │ ├── aws-smithy-http-tower v0.47.0 (*)
│ │ ├── aws-smithy-query v0.47.0
│ │ │ ├── aws-smithy-types v0.47.0 (*)
│ │ ├── aws-smithy-types v0.47.0 (*)
│ │ ├── aws-smithy-xml v0.47.0
│ │ ├── aws-types v0.47.0 (*)
│ ├── aws-smithy-async v0.47.0 (*)
│ ├── aws-smithy-client v0.47.0 (*)
│ ├── aws-smithy-http v0.47.0 (*)
│ ├── aws-smithy-http-tower v0.47.0 (*)
│ ├── aws-smithy-json v0.47.0 (*)
│ ├── aws-smithy-types v0.47.0 (*)
│ ├── aws-types v0.47.0 (*)
├── aws-sdk-dynamodb v0.17.0
│ ├── aws-endpoint v0.47.0 (*)
│ ├── aws-http v0.47.0 (*)
│ ├── aws-sig-auth v0.47.0 (*)
│ ├── aws-smithy-async v0.47.0 (*)
│ ├── aws-smithy-client v0.47.0 (*)
│ ├── aws-smithy-http v0.47.0 (*)
│ ├── aws-smithy-http-tower v0.47.0 (*)
│ ├── aws-smithy-json v0.47.0 (*)
│ ├── aws-smithy-types v0.47.0 (*)
│ ├── aws-types v0.47.0 (*)
```
```

### Environment details (OS name and version, etc.)

x86_64 GNU/Linux

### Logs

_No response_

Contributor guide

Open the contributing guide

Research direction

No repository files or tests are named. Start by running the supplied Rust reproduction with aws-sdk-dynamodb 0.17.0, focusing on Client::clone, tokio::spawn, spawn_blocking, and futures::executor::block_on; done means the blocking and asynchronous scans no longer deadlock, with a regression test or documented reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, rust
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.