aws / aws/aws-sdk-java-v2

Asynchronous DynamoDB Client V2 is slower than V1

Open
#3,422 2 comments 4 reactions 0 assignees View on GitHub
bug p2
Dominant language
Java
Stars
2.6k
Forks
1k
Avg merge
2d 9h
Merged PRs (30d)
51

Description

### Describe the bug

I tried the DynamoDB asynchronous client v. 2.17.271 and noticed than requests runs slower than on version 1.11.698 (synchronous one)

I also wrote a simple tests for both client versions which calls GetItem request and Scan request in a loop and measure the time.
I used the same HTTP settings, tried on both DynamoDBLocal emulator and actual cloud DynamoDB.
The test for async v2 client runs slower than the test for sync v1 both on DynamoDBLocal and cloud one.

The bigger request (Scan with limit 100) time difference is higher than the smalle one (GetItem). But they both runs slower.

I tried different HTTP netty client configuration (OpenSSL for SSL, completion on thread pool and on event loop thread). Results are the same
What else should I look for (maybe turn on some metrics)? to find where the bottleneck is

### Expected Behavior

At least the same performance. And for multiple concurrent requests should be better due to nature of asynchronous requests.

### Current Behavior

Current measurements (on DynamoDBLocal), smaller number is better

`
version 2.17.271

scan sequential 200 total: 933ms consumed 0 rcu/0 wcu
scan sequential percentiles: 90th 7ms, 50th 5ms
scan parallel 200 total: 226ms consumed 0 rcu/0 wcu
scan parallel percentiles: 90th 61ms, 50th 29ms
getitem sequential 200 total: 357ms consumed 0 rcu/0 wcu
getitem sequential percentiles: 90th 3ms, 50th 3ms
getitem parallel 200 total: 103ms consumed 0 rcu/0 wcu
getitem parallel percentiles: 90th 9ms, 50th 4ms

version 1.11.698
scan sequential 200 total: 540ms consumed 0.0 rcu/0.0 wcu
scan sequential percentiles: 90th 4ms, 50th 4ms
scan parallel 200 total: 215ms consumed 0.0 rcu/0.0 wcu
scan parallel percentiles: 90th 45ms, 50th 38ms
getitem sequential 200 total: 225ms consumed 0.0 rcu/0.0 wcu
getitem sequential percentiles: 90th 6ms, 50th 2ms
getitem parallel 200 total: 51ms consumed 0.0 rcu/0.0 wcu
getitem parallel percentiles: 90th 20ms, 50th 6ms

`

### Reproduction Steps

The code is in [v1](https://github.com/pin2t/dynamodb-tests/blob/main/ScanV1Test.java) and [v2](https://github.com/pin2t/dynamodb-tests/blob/main/ScanV2Test.java)

Basically it measures the time for running a loop
```Java
for (int i = 0; i < nrequests; i++) {
stat.measure(() -> {
ScanResult rs = test.client.scan(new ScanRequest(TABLE)
.withConsistentRead(FALSE)
.withLimit(limit)
.withFilterExpression("#nid > :n")
.withExpressionAttributeNames(Collections.singletonMap("#nid", "nid"))
.withExpressionAttributeValues(Collections.singletonMap(":n", new AttributeValue().withN(Long.toString(test.rnd.nextInt(8000)))))
);
if (rs.getCount() == 0 || rs.getCount() > limit) throw new RuntimeException("invalid number of rows returned " + rs.getCount());
});
}

```
or

```Java
for (int i = 0; i < nrequests; i++) {
ScanResponse rs = stat.measure(
() -> test.client.scan(ScanRequest.builder()
.tableName(TABLE)
.limit(limit)
.filterExpression("#nid > :n")
.expressionAttributeNames(Collections.singletonMap("#nid", "nid"))
.expressionAttributeValues(Collections.singletonMap(":n", AttributeValue.fromN(Long.toString(test.rnd.nextInt(8000)))))
.consistentRead(FALSE)
.returnConsumedCapacity(ReturnConsumedCapacity.TOTAL)
.build())
.join()
);
stat.addConsumed(rs.consumedCapacity());
if (rs.count() == 0 || rs.count() > limit) throw new RuntimeException("wrong number of rows returned " + rs.count());
}

```

Client configuration besides authentication

```Java
builder
.withClientConfiguration(new ClientConfiguration()
.withMaxConnections(200)
.withConnectionMaxIdleMillis(60_000)
.withConnectionTimeout(30_000)
.withClientExecutionTimeout(30_000)
.withConnectionTTL(60_000)
.withSocketTimeout(30_000)
.withTcpKeepAlive(true)
.withThrottledRetries(true)
)
.build()
```
```Java
builder
.httpClientBuilder(NettyNioAsyncHttpClient.builder()
.maxConcurrency(200)
.maxPendingConnectionAcquires(10_000)
.connectionMaxIdleTime(Duration.ofSeconds(60))
.connectionTimeout(Duration.ofSeconds(30))
.connectionAcquisitionTimeout(Duration.ofSeconds(30))
.readTimeout(Duration.ofSeconds(30)))
.asyncConfiguration(b -> b.advancedOption(SdkAdvancedAsyncClientOption.FUTURE_COMPLETION_EXECUTOR, Runnable::run))
.build()
```
### Possible Solution

_No response_

### Additional Information/Context

_No response_

### AWS Java SDK version used

2.17.271

### JDK version used

openjdk version "1.8.0_342", openjdk version "11.0.16" 2022-07-19

### Operating System and version

Ubuntu Linux 22.04 x86_64

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.